2023-03-29T13:46:38.684 INFO:root:teuthology version: 0.0.1.dev78+ge372916 2023-03-29T13:46:38.684 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi/7225415 --description crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} --verbose --name matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi -- /home/teuthworker/archive/matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi/7225415/orig.config.yaml 2023-03-29T13:46:38.721 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-03-29T13:46:38.804 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi/7225415 branch: wip-matanb-crimson-only-send-map description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} email: null first_in_suite: false job_id: '7225415' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-matanb-crimson-only-send-map ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore rocksdb cf: false debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 err to stderr: true log flush on exit: true log to file: false log to stderr: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: bluestore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 6c07bf89555b2fa3413019059913f7a1fb12fde5 wait-for-scrub: false ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: crimson sha1: 6c07bf89555b2fa3413019059913f7a1fb12fde5 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: min_in: 4 workunit: branch: wip-matanb-crimson-only-send-map env: CRIMSON_COMPAT: '1' sha1: 6c07bf89555b2fa3413019059913f7a1fb12fde5 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b sha1: 6c07bf89555b2fa3413019059913f7a1fb12fde5 sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-matanb-crimson-only-send-map suite_path: /home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa suite_relpath: qa suite_repo: https://github.com/Matan-B/ceph.git suite_sha1: 6c07bf89555b2fa3413019059913f7a1fb12fde5 targets: smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIadmY4+aNamScujYWIQPsOgfh4+G70+dP8RKTyQRgGoU2n/ITJR08kxjBvxrSYCBG7pkNFBB200De0o8R2O0YZLGd1yJlB1yOjHm3mX4RREJWhz5RTuKZzWfiCAzZpV649ZQVtjrTaK1pwgvn371XkvxLzMKn/2GKF5nigPYTK/W6W7c9TlgVdUCi46l2DodSyr+vEQJSyBYaPYZSnYCdtzDukVw3XK2LLIrTwnIDRfU485RfJG43mcgVFaXUmoOVzJ4VyvMmU55zwP6Wp+fo0adS3pZHGUSwotdAIuVnw0LlTJuxG87kWnaMFHcYfICRF1UbVyG+GJbRGLZaoWNKoOC90KKrhzyzLPC+1amkOvvRFWUxzKhl7ywQVYp/kZrE1u6jD3uV0DyMBaK4kEz72ha1BNrPZDelyBtlWFs8ls8FOgphqT8enGuZUgVPC8al3GCdskGKZ7YSVzvPDMMYbDufEKr3lCyWEw/kMfudWmd1HB5DrNqGHcVUjd3bX8s= smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpnrzCKG18+tqEhffbf8sPsXMFhzfP5DhDqb74O8/7+fI+6sQX5u+kDaj/svsMXGQg7rDzMCG9F8kXmC+Y+tEOV8NOCMfIvmRByjl+sxBrPw9hu4YzorMOTZfdKbxx550yTtwEaxJvoTrv+KReVP/2y8FcTZjfEcBCrsVDijSEudH652aEJIfluyUE6nVfT2b7hVdQRt8k547dqYG0/s+fR36HLno0Ay2wJnrUoWQ2y8mbEcWMiDOyMg2Nuwi/TZ+7ix6O1qxW74Fgu40WeUIIff2PImtsFduyPuC7NR78aypQjOPKhoPNaL2ehB0YIZNqp0C+u5pc2E2sMkYPWv9zeyq/yNmWqtVHyymXhExhNKmG+OcMOglq3kme9O8rID+xoYYfTJ5CzDyFpKy2QG1M5IkN8UCuEfICNf6axe6EC9UO7L6Qh3DmxF0ZPwGEy+GkpQM+7yaN3cQmupkP82M66NbSC2rb17pVJLehfsuXXaVGlbghj3WvbO2Hcj4GZXU= tasks: - install: null - ceph: conf: osd: debug monc: 20 flavor: crimson - thrashosds: ceph_objectstore_tool: false chance_down: 0 chance_inject_pause_short: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - rados: clients: - client.0 objects: 50 op_weights: copy_from: 0 delete: 50 read: 100 rollback: 0 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e3729161eecf8e6d09a4069acbb883cdcd5e9f50 timestamp: 2023-03-29_13:23:53 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.803 2023-03-29T13:46:38.805 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa; will attempt to use it 2023-03-29T13:46:38.805 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks 2023-03-29T13:46:38.805 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-03-29T13:46:38.807 INFO:teuthology.task.internal:Checking packages... 2023-03-29T13:46:38.835 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '6c07bf89555b2fa3413019059913f7a1fb12fde5' 2023-03-29T13:46:38.835 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-03-29T13:46:38.835 INFO:teuthology.packaging:ref: None 2023-03-29T13:46:38.835 INFO:teuthology.packaging:tag: None 2023-03-29T13:46:38.835 INFO:teuthology.packaging:branch: wip-matanb-crimson-only-send-map 2023-03-29T13:46:38.835 INFO:teuthology.packaging:sha1: 6c07bf89555b2fa3413019059913f7a1fb12fde5 2023-03-29T13:46:38.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&ref=wip-matanb-crimson-only-send-map 2023-03-29T13:46:38.975 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3104.g6c07bf89 2023-03-29T13:46:38.975 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-03-29T13:46:38.981 INFO:teuthology.task.internal:no buildpackages task found 2023-03-29T13:46:38.982 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-03-29T13:46:39.019 INFO:teuthology.task.internal:Saving configuration 2023-03-29T13:46:39.033 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-03-29T13:46:39.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-29T13:46:39.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi/7225415', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-29 13:41:40.716164', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEShvR2a0XZ6AyGEfgp8qxvFtCLylJSvR3wkHWb77ngtz4aLnFYHgXvocNo9zzdDQpZPOxW0ob0/k6Qjt8DSlc='} 2023-03-29T13:46:39.095 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi/7225415', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-29 13:41:40.718969', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBODf2Rr0QVnzRkO/SKn9hJsR6y5ngVDkKUF8jDwfpJlTN1t4Aod4Ny0gatUvaX/UQwGe4KXn56oSiTz/qudBSh8='} 2023-03-29T13:46:39.096 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-03-29T13:46:39.102 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2023-03-29T13:46:39.102 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-03-29T13:46:39.102 INFO:teuthology.run_tasks:Running task console_log... 2023-03-29T13:46:39.200 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2c2b031e50>, signals=[15]) 2023-03-29T13:46:39.201 INFO:teuthology.run_tasks:Running task internal.connect... 2023-03-29T13:46:39.206 INFO:teuthology.task.internal:Opening connections... 2023-03-29T13:46:39.206 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2023-03-29T13:46:39.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:46:39.277 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2023-03-29T13:46:39.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:46:39.343 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-03-29T13:46:39.349 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2023-03-29T13:46:39.371 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2023-03-29T13:46:39.371 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2023-03-29T13:46:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream" 2023-03-29T13:46:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8" 2023-03-29T13:46:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2023-03-29T13:46:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2023-03-29T13:46:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8" 2023-03-29T13:46:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8" 2023-03-29T13:46:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-29T13:46:39.434 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2023-03-29T13:46:39.434 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-29T13:46:39.434 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/" 2023-03-29T13:46:39.434 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-29T13:46:39.434 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-29T13:46:39.434 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-29T13:46:39.434 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2023-03-29T13:46:39.463 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-03-29T13:46:39.481 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-03-29T13:46:39.481 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-03-29T13:46:39.539 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2023-03-29T13:46:39.539 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8" 2023-03-29T13:46:39.539 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2023-03-29T13:46:39.539 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-29T13:46:39.540 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-29T13:46:39.541 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-03-29T13:46:39.562 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-03-29T13:46:39.575 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-03-29T13:46:39.595 INFO:teuthology.task.internal:Checking for old test directory... 2023-03-29T13:46:39.595 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2023-03-29T13:46:39.597 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2023-03-29T13:46:39.615 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-03-29T13:46:39.621 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-03-29T13:46:39.621 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2023-03-29T13:46:39.657 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2023-03-29T13:46:39.687 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-03-29T13:46:39.746 INFO:teuthology.run_tasks:Running task kernel... 2023-03-29T13:46:39.762 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-03-29T13:46:39.762 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-03-29T13:46:39.763 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-29T13:46:39.763 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-29T13:46:39.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T13:46:39.780 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-03-29T13:46:39.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T13:46:39.782 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2023-03-29T13:46:39.839 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-408.el8.x86_64 2023-03-29T13:46:39.839 INFO:teuthology.task.kernel:Running kernel on smithi089: 4.18.0-408.el8.x86_64 2023-03-29T13:46:39.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum install -y kernel 2023-03-29T13:46:39.842 INFO:teuthology.orchestra.run.smithi017.stdout:4.18.0-408.el8.x86_64 2023-03-29T13:46:39.842 INFO:teuthology.task.kernel:Running kernel on smithi017: 4.18.0-408.el8.x86_64 2023-03-29T13:46:39.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum install -y kernel 2023-03-29T13:46:43.406 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - AppStream 34 MB/s | 29 MB 00:00 2023-03-29T13:46:43.567 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - AppStream 32 MB/s | 29 MB 00:00 2023-03-29T13:46:49.417 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - BaseOS 39 MB/s | 28 MB 00:00 2023-03-29T13:46:49.690 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - BaseOS 44 MB/s | 28 MB 00:00 2023-03-29T13:46:53.699 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - Extras 172 kB/s | 18 kB 00:00 2023-03-29T13:46:54.102 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - PowerTools 22 MB/s | 5.8 MB 00:00 2023-03-29T13:46:54.305 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - Extras 150 kB/s | 18 kB 00:00 2023-03-29T13:46:54.741 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - PowerTools 20 MB/s | 5.8 MB 00:00 2023-03-29T13:46:55.368 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-03-29T13:46:56.073 INFO:teuthology.orchestra.run.smithi017.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-03-29T13:46:56.893 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 14 MB 00:01 2023-03-29T13:46:58.007 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 14 MB 00:01 2023-03-29T13:47:00.418 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 542 kB/s | 24 kB 00:00 2023-03-29T13:47:01.616 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 465 kB/s | 24 kB 00:00 2023-03-29T13:47:03.675 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-29T13:47:03.675 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-29T13:47:03.855 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T13:47:03.855 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:47:03.856 INFO:teuthology.orchestra.run.smithi089.stdout:Install 3 Packages 2023-03-29T13:47:03.857 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:47:03.857 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 85 M 2023-03-29T13:47:03.857 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 94 M 2023-03-29T13:47:03.857 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T13:47:04.178 INFO:teuthology.orchestra.run.smithi089.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 29 MB/s | 9.2 MB 00:00 2023-03-29T13:47:04.953 INFO:teuthology.orchestra.run.smithi089.stdout:(2/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 31 MB/s | 34 MB 00:01 2023-03-29T13:47:05.072 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-29T13:47:05.072 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-29T13:47:05.255 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T13:47:05.255 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:47:05.255 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T13:47:05.255 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout:Install 3 Packages 2023-03-29T13:47:05.256 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:47:05.257 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 85 M 2023-03-29T13:47:05.257 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 94 M 2023-03-29T13:47:05.257 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T13:47:05.411 INFO:teuthology.orchestra.run.smithi089.stdout:(3/3): kernel-core-4.18.0-448.el8.x86_64.rpm 27 MB/s | 41 MB 00:01 2023-03-29T13:47:05.415 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:47:05.415 INFO:teuthology.orchestra.run.smithi089.stdout:Total 54 MB/s | 85 MB 00:01 2023-03-29T13:47:05.584 INFO:teuthology.orchestra.run.smithi017.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 28 MB/s | 9.2 MB 00:00 2023-03-29T13:47:06.046 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T13:47:06.392 INFO:teuthology.orchestra.run.smithi017.stdout:(2/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 30 MB/s | 34 MB 00:01 2023-03-29T13:47:06.427 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T13:47:06.427 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T13:47:06.776 INFO:teuthology.orchestra.run.smithi017.stdout:(3/3): kernel-core-4.18.0-448.el8.x86_64.rpm 27 MB/s | 41 MB 00:01 2023-03-29T13:47:06.779 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:47:06.779 INFO:teuthology.orchestra.run.smithi017.stdout:Total 56 MB/s | 85 MB 00:01 2023-03-29T13:47:07.440 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T13:47:07.875 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T13:47:07.875 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T13:47:08.441 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T13:47:08.466 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T13:47:09.859 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T13:47:09.898 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T13:47:11.179 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T13:47:11.737 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-29T13:47:12.767 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T13:47:13.353 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-29T13:47:15.482 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-29T13:47:16.011 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-29T13:47:17.264 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-29T13:47:17.703 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-29T13:47:19.331 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-29T13:47:19.644 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:21.255 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-29T13:47:21.507 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:41.458 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:41.484 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:41.897 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:41.897 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-29T13:47:41.897 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-29T13:47:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:47:42.499 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T13:47:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-29T13:47:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-29T13:47:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:47:42.499 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T13:47:42.614 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel --last | head -n 1 2023-03-29T13:47:42.811 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-448.el8.x86_64 Wed 29 Mar 2023 01:47:19 PM UTC 2023-03-29T13:47:42.813 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-29T13:47:42.813 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-29T13:47:42.813 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-29T13:47:42.813 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-29T13:47:42.813 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2023-03-29T13:47:42.813 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=None, version=distro) 2023-03-29T13:47:42.813 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel --last | head -n 1 2023-03-29T13:47:43.001 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-448.el8.x86_64 Wed 29 Mar 2023 01:47:19 PM UTC 2023-03-29T13:47:43.228 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-29T13:47:43.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -qi grub2-tools 2023-03-29T13:47:43.262 INFO:teuthology.orchestra.run.smithi089.stdout:Name : grub2-tools 2023-03-29T13:47:43.262 INFO:teuthology.orchestra.run.smithi089.stdout:Epoch : 1 2023-03-29T13:47:43.262 INFO:teuthology.orchestra.run.smithi089.stdout:Version : 2.02 2023-03-29T13:47:43.262 INFO:teuthology.orchestra.run.smithi089.stdout:Release : 129.el8 2023-03-29T13:47:43.262 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: x86_64 2023-03-29T13:47:43.262 INFO:teuthology.orchestra.run.smithi089.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-29T13:47:43.262 INFO:teuthology.orchestra.run.smithi089.stdout:Group : System Environment/Base 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Size : 9265586 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:License : GPLv3+ 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Relocations : (not relocatable) 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Packager : CentOS BuildSystem 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Vendor : CentOS 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Summary : Support tools for GRUB. 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:Description : 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout:hardware devices. 2023-03-29T13:47:43.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:47:43.264 INFO:teuthology.orchestra.run.smithi089.stdout:This subpackage provides tools for support of all platforms. 2023-03-29T13:47:43.264 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-29T13:47:43.264 INFO:teuthology.task.kernel:Updating grub on smithi089 to boot 4.18.0-448.el8.x86_64 2023-03-29T13:47:43.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-29T13:47:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2023-03-29T13:47:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:45.593 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:45.593 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-29T13:47:45.594 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-29T13:47:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-29T13:47:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:47:46.289 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T13:47:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-29T13:47:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-29T13:47:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:47:46.289 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T13:47:46.401 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel --last | head -n 1 2023-03-29T13:47:46.645 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-4.18.0-448.el8.x86_64 Wed 29 Mar 2023 01:47:21 PM UTC 2023-03-29T13:47:46.647 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-29T13:47:46.647 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-29T13:47:46.647 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-29T13:47:46.647 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-29T13:47:46.647 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-03-29T13:47:46.647 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=None, version=distro) 2023-03-29T13:47:46.647 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel --last | head -n 1 2023-03-29T13:47:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:done 2023-03-29T13:47:46.707 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-03-29T13:47:46.730 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.wxpXENZVYq 2023-03-29T13:47:46.730 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wxpXENZVYq 2023-03-29T13:47:46.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.wxpXENZVYq 2023-03-29T13:47:46.842 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-4.18.0-448.el8.x86_64 Wed 29 Mar 2023 01:47:21 PM UTC 2023-03-29T13:47:46.844 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-29T13:47:46.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo rpm -qi grub2-tools 2023-03-29T13:47:46.877 INFO:teuthology.orchestra.run.smithi017.stdout:Name : grub2-tools 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Epoch : 1 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Version : 2.02 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Release : 129.el8 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: x86_64 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Group : System Environment/Base 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Size : 9265586 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:License : GPLv3+ 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-29T13:47:46.878 INFO:teuthology.orchestra.run.smithi017.stdout:Relocations : (not relocatable) 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:Packager : CentOS BuildSystem 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:Vendor : CentOS 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:Summary : Support tools for GRUB. 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:Description : 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:hardware devices. 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:47:46.879 INFO:teuthology.orchestra.run.smithi017.stdout:This subpackage provides tools for support of all platforms. 2023-03-29T13:47:46.880 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-29T13:47:46.880 INFO:teuthology.task.kernel:Updating grub on smithi017 to boot 4.18.0-448.el8.x86_64 2023-03-29T13:47:46.880 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-29T13:47:46.951 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.wxpXENZVYq is 6KB 2023-03-29T13:47:46.998 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.wxpXENZVYq 2023-03-29T13:47:47.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo /bin/ls /boot/loader/entries || true 2023-03-29T13:47:47.080 INFO:teuthology.orchestra.run.smithi089.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-29T13:47:47.080 INFO:teuthology.orchestra.run.smithi089.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-29T13:47:47.080 INFO:teuthology.orchestra.run.smithi089.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-29T13:47:47.080 INFO:teuthology.orchestra.run.smithi089.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-29T13:47:47.081 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-29T13:47:47.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2023-03-29T13:47:48.204 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2023-03-29T13:47:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:done 2023-03-29T13:47:50.293 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2023-03-29T13:47:50.313 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.WUJjKw8X1n 2023-03-29T13:47:50.313 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WUJjKw8X1n 2023-03-29T13:47:50.393 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.WUJjKw8X1n 2023-03-29T13:47:50.544 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.WUJjKw8X1n is 6KB 2023-03-29T13:47:50.592 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.WUJjKw8X1n 2023-03-29T13:47:50.609 DEBUG:teuthology.orchestra.run.smithi017:> sudo /bin/ls /boot/loader/entries || true 2023-03-29T13:47:50.679 INFO:teuthology.orchestra.run.smithi017.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-29T13:47:50.679 INFO:teuthology.orchestra.run.smithi017.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-29T13:47:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-29T13:47:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-29T13:47:50.680 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-29T13:47:50.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2023-03-29T13:48:17.270 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2023-03-29T13:48:17.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-03-29T13:48:17.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:48:20.861 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-03-29T13:48:20.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-03-29T13:48:20.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:48:35.721 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-03-29T13:48:39.305 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.17 2023-03-29T13:49:05.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-03-29T13:49:05.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:49:08.812 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-03-29T13:49:09.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-03-29T13:49:09.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:49:25.534 DEBUG:teuthology.orchestra.run.smithi017:> true 2023-03-29T13:49:25.880 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-03-29T13:49:25.880 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-03-29T13:49:25.880 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2023-03-29T13:49:25.898 INFO:teuthology.orchestra.run.smithi017.stdout:4.18.0-448.el8.x86_64 2023-03-29T13:49:25.898 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-29T13:49:25.898 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-29T13:49:25.899 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-29T13:49:26.899 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-03-29T13:49:26.900 DEBUG:teuthology.orchestra.run.smithi017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-29T13:49:26.988 INFO:teuthology.orchestra.run.smithi017.stdout:ttyS1 2023-03-29T13:49:27.062 DEBUG:teuthology.parallel:result is None 2023-03-29T13:49:38.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-03-29T13:49:38.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:49:39.086 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-03-29T13:49:39.382 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-03-29T13:49:39.382 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "4.18.0-448.el8.x86_64"... 2023-03-29T13:49:39.382 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-03-29T13:49:39.442 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-448.el8.x86_64 2023-03-29T13:49:39.442 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-29T13:49:39.442 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-29T13:49:39.443 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-29T13:49:40.443 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2023-03-29T13:49:40.444 DEBUG:teuthology.orchestra.run.smithi089:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-29T13:49:40.535 INFO:teuthology.orchestra.run.smithi089.stdout:ttyS1 2023-03-29T13:49:40.554 DEBUG:teuthology.parallel:result is None 2023-03-29T13:49:40.555 INFO:teuthology.run_tasks:Running task internal.base... 2023-03-29T13:49:40.562 INFO:teuthology.task.internal:Creating test directory... 2023-03-29T13:49:40.562 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-29T13:49:40.566 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-29T13:49:40.588 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-03-29T13:49:40.594 INFO:teuthology.run_tasks:Running task internal.archive... 2023-03-29T13:49:40.631 INFO:teuthology.task.internal:Creating archive directory... 2023-03-29T13:49:40.631 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-29T13:49:40.633 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-29T13:49:40.669 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-03-29T13:49:40.676 INFO:teuthology.task.internal:Enabling coredump saving... 2023-03-29T13:49:40.676 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-29T13:49:40.706 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-29T13:49:40.743 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T13:49:40.744 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T13:49:40.754 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T13:49:40.755 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T13:49:40.756 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-03-29T13:49:40.763 INFO:teuthology.task.internal:Configuring sudo... 2023-03-29T13:49:40.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-29T13:49:40.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-29T13:49:40.829 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-03-29T13:49:40.838 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-03-29T13:49:40.839 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-29T13:49:40.871 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-29T13:49:40.890 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T13:49:40.983 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T13:49:41.029 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T13:49:41.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T13:49:41.160 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T13:49:41.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-29T13:49:41.229 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T13:49:41.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T13:49:41.326 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T13:49:41.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T13:49:41.456 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T13:49:41.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-29T13:49:41.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2023-03-29T13:49:41.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2023-03-29T13:49:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-29T13:49:41.611 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-29T13:49:42.062 INFO:teuthology.run_tasks:Running task internal.timer... 2023-03-29T13:49:42.069 INFO:teuthology.task.internal:Starting timer... 2023-03-29T13:49:42.069 INFO:teuthology.run_tasks:Running task pcp... 2023-03-29T13:49:42.118 INFO:teuthology.run_tasks:Running task selinux... 2023-03-29T13:49:42.149 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-03-29T13:49:42.150 DEBUG:teuthology.orchestra.run.smithi017:> sudo service auditd rotate 2023-03-29T13:49:42.219 INFO:teuthology.orchestra.run.smithi017.stdout:Rotating logs: 2023-03-29T13:49:42.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2023-03-29T13:49:42.282 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2023-03-29T13:49:42.284 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-03-29T13:49:42.284 DEBUG:teuthology.orchestra.run.smithi017:> /usr/sbin/getenforce 2023-03-29T13:49:42.306 INFO:teuthology.orchestra.run.smithi017.stdout:Permissive 2023-03-29T13:49:42.306 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2023-03-29T13:49:42.343 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2023-03-29T13:49:42.344 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi017.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi089.front.sepia.ceph.com': 'permissive'} 2023-03-29T13:49:42.344 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-29T13:49:42.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T13:49:42.375 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-29T13:49:42.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T13:49:42.412 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-03-29T13:49:42.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo /usr/sbin/setenforce permissive 2023-03-29T13:49:42.443 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2023-03-29T13:49:42.490 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-03-29T13:49:42.502 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-03-29T13:49:42.508 INFO:teuthology.repo_utils:Fetching main from origin 2023-03-29T13:49:42.638 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-03-29T13:49:42.656 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-03-29T13:49:42.658 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi089.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-03-29T13:57:00.781 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi089.front.sepia.ceph.com')] 2023-03-29T13:57:00.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-03-29T13:57:00.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:57:00.852 DEBUG:teuthology.orchestra.run.smithi017:> true 2023-03-29T13:57:00.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-03-29T13:57:00.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T13:57:00.992 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-03-29T13:57:01.060 INFO:teuthology.run_tasks:Running task clock... 2023-03-29T13:57:01.072 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-03-29T13:57:01.073 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-29T13:57:01.073 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-29T13:57:01.075 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-29T13:57:01.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-29T13:57:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-29T13:57:01.120 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-29T13:57:01.134 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-29T13:57:01.149 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-29T13:57:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: ntpd: command not found 2023-03-29T13:57:01.207 INFO:teuthology.orchestra.run.smithi017.stdout:506 Cannot talk to daemon 2023-03-29T13:57:01.213 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2023-03-29T13:57:01.222 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-29T13:57:01.225 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2023-03-29T13:57:01.236 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-29T13:57:01.239 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-29T13:57:01.252 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-29T13:57:01.306 INFO:teuthology.orchestra.run.smithi017.stderr:bash: ntpq: command not found 2023-03-29T13:57:01.309 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-29T13:57:01.310 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2023-03-29T13:57:01.310 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.310 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.310 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.310 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.323 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-03-29T13:57:01.327 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-29T13:57:01.327 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-03-29T13:57:01.327 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.327 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.327 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.327 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T13:57:01.328 INFO:teuthology.run_tasks:Running task install... 2023-03-29T13:57:01.336 DEBUG:teuthology.task.install:project ceph 2023-03-29T13:57:01.336 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5'}} 2023-03-29T13:57:01.336 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5'} 2023-03-29T13:57:01.336 INFO:teuthology.task.install:Using flavor: crimson 2023-03-29T13:57:01.341 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-03-29T13:57:01.342 INFO:teuthology.task.install:extra packages: [] 2023-03-29T13:57:01.342 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5', 'tag': None, 'wait_for_package': False} 2023-03-29T13:57:01.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=6c07bf89555b2fa3413019059913f7a1fb12fde5 2023-03-29T13:57:01.345 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5', 'tag': None, 'wait_for_package': False} 2023-03-29T13:57:01.345 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=6c07bf89555b2fa3413019059913f7a1fb12fde5 2023-03-29T13:57:01.483 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/ 2023-03-29T13:57:01.484 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3104.g6c07bf89 2023-03-29T13:57:01.491 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/ 2023-03-29T13:57:01.491 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3104.g6c07bf89 2023-03-29T13:57:01.591 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-29T13:57:01.591 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T13:57:01.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-29T13:57:01.598 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-only-send-map/6c07bf89555b2fa3413019059913f7a1fb12fde5/centos/8/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-29T13:57:01.599 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T13:57:01.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-29T13:57:01.620 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-03-29T13:57:01.620 DEBUG:teuthology.orchestra.run.smithi017:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c07bf89555b2fa3413019059913f7a1fb12fde5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-29T13:57:01.628 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-03-29T13:57:01.628 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c07bf89555b2fa3413019059913f7a1fb12fde5/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-29T13:57:01.690 DEBUG:teuthology.orchestra.run.smithi017:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-29T13:57:01.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-29T13:57:01.770 DEBUG:teuthology.orchestra.run.smithi017:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-29T13:57:01.772 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-29T13:57:01.838 INFO:teuthology.orchestra.run.smithi017.stdout:check_obsoletes = 1 2023-03-29T13:57:01.839 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum clean all 2023-03-29T13:57:01.840 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2023-03-29T13:57:01.841 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2023-03-29T13:57:02.093 INFO:teuthology.orchestra.run.smithi089.stdout:71 files removed 2023-03-29T13:57:02.103 INFO:teuthology.orchestra.run.smithi017.stdout:71 files removed 2023-03-29T13:57:02.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw 2023-03-29T13:57:02.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-radosgw 2023-03-29T13:57:02.782 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 322 kB/s | 83 kB 00:00 2023-03-29T13:57:02.817 INFO:teuthology.orchestra.run.smithi017.stdout:ceph packages for x86_64 326 kB/s | 83 kB 00:00 2023-03-29T13:57:03.075 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 37 kB/s | 9.4 kB 00:00 2023-03-29T13:57:03.092 INFO:teuthology.orchestra.run.smithi017.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2023-03-29T13:57:03.334 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 6.9 kB/s | 1.6 kB 00:00 2023-03-29T13:57:03.352 INFO:teuthology.orchestra.run.smithi017.stdout:ceph source packages 6.9 kB/s | 1.6 kB 00:00 2023-03-29T13:57:04.128 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - AppStream 37 MB/s | 29 MB 00:00 2023-03-29T13:57:04.246 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - AppStream 33 MB/s | 29 MB 00:00 2023-03-29T13:57:09.912 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - BaseOS 43 MB/s | 28 MB 00:00 2023-03-29T13:57:10.438 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - BaseOS 45 MB/s | 28 MB 00:00 2023-03-29T13:57:14.244 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - Extras 151 kB/s | 18 kB 00:00 2023-03-29T13:57:14.647 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - PowerTools 21 MB/s | 5.8 MB 00:00 2023-03-29T13:57:15.095 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - Extras 142 kB/s | 18 kB 00:00 2023-03-29T13:57:15.498 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-8 - PowerTools 23 MB/s | 5.8 MB 00:00 2023-03-29T13:57:15.929 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-03-29T13:57:16.814 INFO:teuthology.orchestra.run.smithi017.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-03-29T13:57:17.489 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 14 MB 00:01 2023-03-29T13:57:18.273 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 11 MB/s | 14 MB 00:01 2023-03-29T13:57:20.830 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 410 kB/s | 24 kB 00:00 2023-03-29T13:57:21.998 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 133 kB/s | 24 kB 00:00 2023-03-29T13:57:24.431 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T13:57:24.433 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 14 M 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 3.3 M 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 3.6 M 2023-03-29T13:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 6.8 M 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 23 M 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 24 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 744 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 462 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 7.2 M 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 45 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 117 k 2023-03-29T13:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 190 k 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 371 k 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 365 k 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 107 k 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout:Install 21 Packages 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 2 Packages 2023-03-29T13:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:57:24.437 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 63 M 2023-03-29T13:57:24.437 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T13:57:25.365 INFO:teuthology.orchestra.run.smithi089.stdout:(1/23): ceph-base-18.0.0-3104.g6c07bf89.el8.x86 7.3 MB/s | 6.8 MB 00:00 2023-03-29T13:57:25.433 INFO:teuthology.orchestra.run.smithi089.stdout:(2/23): ceph-selinux-18.0.0-3104.g6c07bf89.el8. 363 kB/s | 24 kB 00:00 2023-03-29T13:57:25.600 INFO:teuthology.orchestra.run.smithi089.stdout:(3/23): libcephfs2-18.0.0-3104.g6c07bf89.el8.x8 4.4 MB/s | 744 kB 00:00 2023-03-29T13:57:25.708 INFO:teuthology.orchestra.run.smithi089.stdout:(4/23): libradosstriper1-18.0.0-3104.g6c07bf89. 4.2 MB/s | 462 kB 00:00 2023-03-29T13:57:25.885 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T13:57:25.888 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:57:25.888 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T13:57:25.888 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:57:25.888 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T13:57:25.888 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 14 M 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrading: 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 3.3 M 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 3.6 M 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 6.8 M 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 23 M 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 24 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 744 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 462 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 7.2 M 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-29T13:57:25.889 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 45 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 117 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 190 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 371 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 365 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 107 k 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout:Install 21 Packages 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrade 2 Packages 2023-03-29T13:57:25.890 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:57:25.891 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 63 M 2023-03-29T13:57:25.891 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T13:57:26.075 INFO:teuthology.orchestra.run.smithi089.stdout:(5/23): ceph-radosgw-18.0.0-3104.g6c07bf89.el8. 8.8 MB/s | 14 MB 00:01 2023-03-29T13:57:26.142 INFO:teuthology.orchestra.run.smithi089.stdout:(6/23): python3-ceph-argparse-18.0.0-3104.g6c07 672 kB/s | 45 kB 00:00 2023-03-29T13:57:26.226 INFO:teuthology.orchestra.run.smithi089.stdout:(7/23): python3-ceph-common-18.0.0-3104.g6c07bf 1.4 MB/s | 117 kB 00:00 2023-03-29T13:57:26.318 INFO:teuthology.orchestra.run.smithi089.stdout:(8/23): python3-cephfs-18.0.0-3104.g6c07bf89.el 2.0 MB/s | 190 kB 00:00 2023-03-29T13:57:26.426 INFO:teuthology.orchestra.run.smithi089.stdout:(9/23): python3-rados-18.0.0-3104.g6c07bf89.el8 3.3 MB/s | 371 kB 00:00 2023-03-29T13:57:26.527 INFO:teuthology.orchestra.run.smithi089.stdout:(10/23): python3-rbd-18.0.0-3104.g6c07bf89.el8. 3.6 MB/s | 365 kB 00:00 2023-03-29T13:57:26.677 INFO:teuthology.orchestra.run.smithi089.stdout:(11/23): librgw2-18.0.0-3104.g6c07bf89.el8.x86_ 7.4 MB/s | 7.2 MB 00:00 2023-03-29T13:57:26.702 INFO:teuthology.orchestra.run.smithi089.stdout:(12/23): python3-rgw-18.0.0-3104.g6c07bf89.el8. 609 kB/s | 107 kB 00:00 2023-03-29T13:57:26.928 INFO:teuthology.orchestra.run.smithi089.stdout:(13/23): ceph-common-18.0.0-3104.g6c07bf89.el8. 9.2 MB/s | 23 MB 00:02 2023-03-29T13:57:26.970 INFO:teuthology.orchestra.run.smithi089.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 988 kB/s | 259 kB 00:00 2023-03-29T13:57:26.995 INFO:teuthology.orchestra.run.smithi089.stdout:(15/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.1 MB/s | 354 kB 00:00 2023-03-29T13:57:27.012 INFO:teuthology.orchestra.run.smithi089.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-03-29T13:57:27.029 INFO:teuthology.orchestra.run.smithi089.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 453 kB/s | 44 kB 00:00 2023-03-29T13:57:27.154 INFO:teuthology.orchestra.run.smithi089.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 414 kB/s | 59 kB 00:00 2023-03-29T13:57:27.179 INFO:teuthology.orchestra.run.smithi089.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 499 kB/s | 75 kB 00:00 2023-03-29T13:57:27.230 INFO:teuthology.orchestra.run.smithi089.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-03-29T13:57:27.480 INFO:teuthology.orchestra.run.smithi089.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2023-03-29T13:57:27.600 INFO:teuthology.orchestra.run.smithi017.stdout:(1/23): ceph-base-18.0.0-3104.g6c07bf89.el8.x86 4.0 MB/s | 6.8 MB 00:01 2023-03-29T13:57:27.683 INFO:teuthology.orchestra.run.smithi017.stdout:(2/23): ceph-selinux-18.0.0-3104.g6c07bf89.el8. 296 kB/s | 24 kB 00:00 2023-03-29T13:57:27.872 INFO:teuthology.orchestra.run.smithi089.stdout:(22/23): librados2-18.0.0-3104.g6c07bf89.el8.x8 4.8 MB/s | 3.3 MB 00:00 2023-03-29T13:57:27.900 INFO:teuthology.orchestra.run.smithi017.stdout:(3/23): libcephfs2-18.0.0-3104.g6c07bf89.el8.x8 3.4 MB/s | 744 kB 00:00 2023-03-29T13:57:27.939 INFO:teuthology.orchestra.run.smithi089.stdout:(23/23): librbd1-18.0.0-3104.g6c07bf89.el8.x86_ 5.0 MB/s | 3.6 MB 00:00 2023-03-29T13:57:27.943 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:57:27.943 INFO:teuthology.orchestra.run.smithi089.stdout:Total 18 MB/s | 63 MB 00:03 2023-03-29T13:57:27.985 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T13:57:28.025 INFO:teuthology.orchestra.run.smithi017.stdout:(4/23): libradosstriper1-18.0.0-3104.g6c07bf89. 3.6 MB/s | 462 kB 00:00 2023-03-29T13:57:28.031 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T13:57:28.031 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T13:57:28.419 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T13:57:28.422 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T13:57:28.484 INFO:teuthology.orchestra.run.smithi017.stdout:(5/23): ceph-common-18.0.0-3104.g6c07bf89.el8.x 8.9 MB/s | 23 MB 00:02 2023-03-29T13:57:28.534 INFO:teuthology.orchestra.run.smithi017.stdout:(6/23): python3-ceph-argparse-18.0.0-3104.g6c07 895 kB/s | 45 kB 00:00 2023-03-29T13:57:28.584 INFO:teuthology.orchestra.run.smithi017.stdout:(7/23): python3-ceph-common-18.0.0-3104.g6c07bf 2.3 MB/s | 117 kB 00:00 2023-03-29T13:57:28.744 INFO:teuthology.orchestra.run.smithi017.stdout:(8/23): ceph-radosgw-18.0.0-3104.g6c07bf89.el8. 5.1 MB/s | 14 MB 00:02 2023-03-29T13:57:28.777 INFO:teuthology.orchestra.run.smithi017.stdout:(9/23): python3-cephfs-18.0.0-3104.g6c07bf89.el 991 kB/s | 190 kB 00:00 2023-03-29T13:57:28.827 INFO:teuthology.orchestra.run.smithi017.stdout:(10/23): python3-rados-18.0.0-3104.g6c07bf89.el 4.4 MB/s | 371 kB 00:00 2023-03-29T13:57:28.852 INFO:teuthology.orchestra.run.smithi017.stdout:(11/23): python3-rbd-18.0.0-3104.g6c07bf89.el8. 4.8 MB/s | 365 kB 00:00 2023-03-29T13:57:28.869 INFO:teuthology.orchestra.run.smithi017.stdout:(12/23): python3-rgw-18.0.0-3104.g6c07bf89.el8. 2.5 MB/s | 107 kB 00:00 2023-03-29T13:57:28.928 INFO:teuthology.orchestra.run.smithi017.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2023-03-29T13:57:28.953 INFO:teuthology.orchestra.run.smithi017.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2023-03-29T13:57:28.978 INFO:teuthology.orchestra.run.smithi017.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.4 MB/s | 259 kB 00:00 2023-03-29T13:57:29.071 INFO:teuthology.orchestra.run.smithi017.stdout:(16/23): librgw2-18.0.0-3104.g6c07bf89.el8.x86_ 6.9 MB/s | 7.2 MB 00:01 2023-03-29T13:57:29.096 INFO:teuthology.orchestra.run.smithi017.stdout:(17/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 329 kB/s | 47 kB 00:00 2023-03-29T13:57:29.238 INFO:teuthology.orchestra.run.smithi017.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 415 kB/s | 59 kB 00:00 2023-03-29T13:57:29.263 INFO:teuthology.orchestra.run.smithi017.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 450 kB/s | 75 kB 00:00 2023-03-29T13:57:29.313 INFO:teuthology.orchestra.run.smithi017.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 915 kB/s | 306 kB 00:00 2023-03-29T13:57:29.502 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T13:57:29.564 INFO:teuthology.orchestra.run.smithi017.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2023-03-29T13:57:29.622 INFO:teuthology.orchestra.run.smithi017.stdout:(22/23): librbd1-18.0.0-3104.g6c07bf89.el8.x86_ 12 MB/s | 3.6 MB 00:00 2023-03-29T13:57:29.681 INFO:teuthology.orchestra.run.smithi017.stdout:(23/23): librados2-18.0.0-3104.g6c07bf89.el8.x8 8.0 MB/s | 3.3 MB 00:00 2023-03-29T13:57:29.685 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:57:29.685 INFO:teuthology.orchestra.run.smithi017.stdout:Total 17 MB/s | 63 MB 00:03 2023-03-29T13:57:29.728 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T13:57:29.735 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-03-29T13:57:29.777 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T13:57:29.777 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T13:57:29.912 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-03-29T13:57:30.196 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T13:57:30.199 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T13:57:30.292 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-03-29T13:57:30.504 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/25 2023-03-29T13:57:30.602 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/25 2023-03-29T13:57:30.859 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 4/25 2023-03-29T13:57:31.070 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/25 2023-03-29T13:57:31.271 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T13:57:31.490 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-03-29T13:57:31.491 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/25 2023-03-29T13:57:31.662 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 6/25 2023-03-29T13:57:31.694 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-03-29T13:57:31.888 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 6/25 2023-03-29T13:57:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-03-29T13:57:32.110 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-03-29T13:57:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-03-29T13:57:32.335 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/25 2023-03-29T13:57:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-29T13:57:32.440 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/25 2023-03-29T13:57:32.491 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-29T13:57:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:18.0.0-3104.g6c07bf89.el8. 10/25 2023-03-29T13:57:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 4/25 2023-03-29T13:57:32.839 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/25 2023-03-29T13:57:32.860 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/25 2023-03-29T13:57:33.080 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 12/25 2023-03-29T13:57:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_6 13/25 2023-03-29T13:57:33.297 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_6 13/25 2023-03-29T13:57:33.306 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/25 2023-03-29T13:57:33.493 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 6/25 2023-03-29T13:57:33.503 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-03-29T13:57:33.709 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 6/25 2023-03-29T13:57:33.881 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-03-29T13:57:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-03-29T13:57:34.176 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-29T13:57:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-29T13:57:34.379 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-argparse-2:18.0.0-3104.g6c07bf89.el8. 10/25 2023-03-29T13:57:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-03-29T13:57:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 16/25 2023-03-29T13:57:34.623 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/25 2023-03-29T13:57:34.677 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 16/25 2023-03-29T13:57:34.856 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 12/25 2023-03-29T13:57:34.894 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 17/25 2023-03-29T13:57:34.993 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_6 13/25 2023-03-29T13:57:35.046 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_6 13/25 2023-03-29T13:57:35.077 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-03-29T13:57:35.238 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-03-29T13:57:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:18.0.0-3104.g6c07bf89.el8.x8 19/25 2023-03-29T13:57:36.186 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-03-29T13:57:36.351 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 16/25 2023-03-29T13:57:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 16/25 2023-03-29T13:57:36.593 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 17/25 2023-03-29T13:57:36.754 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-03-29T13:57:37.026 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-common-2:18.0.0-3104.g6c07bf89.el8.x8 19/25 2023-03-29T13:57:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 20/25 2023-03-29T13:57:39.336 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 20/25 2023-03-29T13:57:40.126 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 20/25 2023-03-29T13:57:40.396 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 21/25 2023-03-29T13:57:40.504 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 21/25 2023-03-29T13:57:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 22/25 2023-03-29T13:57:41.045 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 20/25 2023-03-29T13:57:41.260 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 20/25 2023-03-29T13:57:42.092 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 20/25 2023-03-29T13:57:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 21/25 2023-03-29T13:57:42.470 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 21/25 2023-03-29T13:57:42.642 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 22/25 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 22/25 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /run 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2023-03-29T13:58:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2023-03-29T13:58:00.137 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-03-29T13:58:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:01.743 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 23/25 2023-03-29T13:58:01.763 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 23/25 2023-03-29T13:58:01.764 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-29T13:58:01.976 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-29T13:58:01.976 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-29T13:58:02.683 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 4/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_6 6/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 7/25 2023-03-29T13:58:02.684 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3104.g6c07bf89.el8. 8/25 2023-03-29T13:58:02.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:18.0.0-3104.g6c07bf89.el8.x8 9/25 2023-03-29T13:58:02.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 10/25 2023-03-29T13:58:02.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/25 2023-03-29T13:58:02.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 12/25 2023-03-29T13:58:02.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 13/25 2023-03-29T13:58:02.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-03-29T13:58:02.685 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-03-29T13:58:02.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-03-29T13:58:02.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-03-29T13:58:02.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-03-29T13:58:02.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-03-29T13:58:02.686 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-03-29T13:58:02.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-03-29T13:58:02.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 22/25 2023-03-29T13:58:02.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-03-29T13:58:02.687 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 24/25 2023-03-29T13:58:02.773 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 22/25 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /sys 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /proc 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /dev 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /run 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /mnt 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /var/tmp 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /home 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /tmp 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /dev 2023-03-29T13:58:02.774 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:03.531 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-03-29T13:58:03.531 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:03.531 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-03-29T13:58:03.531 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.531 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.532 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-29T13:58:03.533 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.533 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.533 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:03.533 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-29T13:58:03.533 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:03.533 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T13:58:03.801 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-test 2023-03-29T13:58:04.441 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 23/25 2023-03-29T13:58:04.463 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 23/25 2023-03-29T13:58:04.463 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-29T13:58:04.549 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:44 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T13:58:04.700 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-29T13:58:04.700 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-29T13:58:05.424 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-29T13:58:05.424 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/25 2023-03-29T13:58:05.424 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/25 2023-03-29T13:58:05.424 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/25 2023-03-29T13:58:05.424 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 4/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_6 6/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 7/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3104.g6c07bf89.el8. 8/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-common-2:18.0.0-3104.g6c07bf89.el8.x8 9/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 10/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 12/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 13/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-03-29T13:58:05.425 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-03-29T13:58:05.426 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 22/25 2023-03-29T13:58:05.426 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-03-29T13:58:05.426 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 24/25 2023-03-29T13:58:05.765 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 142 M 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: cryptopp x86_64 8.6.0-1.el8 epel 1.3 M 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: libasan6 x86_64 11.2.1-9.1.el8 CentOS-AppStream 402 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: libatomic x86_64 8.5.0-18.el8 CentOS-BaseOS 25 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 135 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: libubsan x86_64 8.5.0-18.el8 CentOS-BaseOS 152 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-29T13:58:05.767 INFO:teuthology.orchestra.run.smithi089.stdout: yaml-cpp x86_64 0.6.2-3.el8 epel 171 k 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:Install 11 Packages 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 145 M 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 1.6 G 2023-03-29T13:58:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T13:58:05.840 INFO:teuthology.orchestra.run.smithi089.stdout:(1/11): jq-1.6-6.el8.x86_64.rpm 2.8 MB/s | 202 kB 00:00 2023-03-29T13:58:05.874 INFO:teuthology.orchestra.run.smithi089.stdout:(2/11): libasan6-11.2.1-9.1.el8.x86_64.rpm 12 MB/s | 402 kB 00:00 2023-03-29T13:58:05.899 INFO:teuthology.orchestra.run.smithi089.stdout:(3/11): oniguruma-6.8.2-2.el8.x86_64.rpm 7.3 MB/s | 187 kB 00:00 2023-03-29T13:58:05.933 INFO:teuthology.orchestra.run.smithi089.stdout:(4/11): socat-1.7.4.1-1.el8.x86_64.rpm 9.5 MB/s | 323 kB 00:00 2023-03-29T13:58:05.958 INFO:teuthology.orchestra.run.smithi089.stdout:(5/11): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2023-03-29T13:58:05.984 INFO:teuthology.orchestra.run.smithi089.stdout:(6/11): libatomic-8.5.0-18.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2023-03-29T13:58:06.000 INFO:teuthology.orchestra.run.smithi089.stdout:(7/11): libcephsqlite-18.0.0-3104.g6c07bf89.el8 586 kB/s | 135 kB 00:00 2023-03-29T13:58:06.026 INFO:teuthology.orchestra.run.smithi089.stdout:(8/11): libubsan-8.5.0-18.el8.x86_64.rpm 3.5 MB/s | 152 kB 00:00 2023-03-29T13:58:06.201 INFO:teuthology.orchestra.run.smithi089.stdout:(9/11): yaml-cpp-0.6.2-3.el8.x86_64.rpm 974 kB/s | 171 kB 00:00 2023-03-29T13:58:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout:Upgraded: 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-29T13:58:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:06.341 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T13:58:06.510 INFO:teuthology.orchestra.run.smithi089.stdout:(10/11): cryptopp-8.6.0-1.el8.x86_64.rpm 2.5 MB/s | 1.3 MB 00:00 2023-03-29T13:58:06.634 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-test 2023-03-29T13:58:07.409 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:46 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T13:58:08.698 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 142 M 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: cryptopp x86_64 8.6.0-1.el8 epel 1.3 M 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: libasan6 x86_64 11.2.1-9.1.el8 CentOS-AppStream 402 k 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: libatomic x86_64 8.5.0-18.el8 CentOS-BaseOS 25 k 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 135 k 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: libubsan x86_64 8.5.0-18.el8 CentOS-BaseOS 152 k 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-29T13:58:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: yaml-cpp x86_64 0.6.2-3.el8 epel 171 k 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:Install 11 Packages 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 145 M 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 1.6 G 2023-03-29T13:58:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T13:58:08.777 INFO:teuthology.orchestra.run.smithi017.stdout:(1/11): jq-1.6-6.el8.x86_64.rpm 2.7 MB/s | 202 kB 00:00 2023-03-29T13:58:08.819 INFO:teuthology.orchestra.run.smithi017.stdout:(2/11): libasan6-11.2.1-9.1.el8.x86_64.rpm 12 MB/s | 402 kB 00:00 2023-03-29T13:58:08.844 INFO:teuthology.orchestra.run.smithi017.stdout:(3/11): oniguruma-6.8.2-2.el8.x86_64.rpm 7.3 MB/s | 187 kB 00:00 2023-03-29T13:58:08.878 INFO:teuthology.orchestra.run.smithi017.stdout:(4/11): socat-1.7.4.1-1.el8.x86_64.rpm 9.7 MB/s | 323 kB 00:00 2023-03-29T13:58:08.903 INFO:teuthology.orchestra.run.smithi017.stdout:(5/11): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-03-29T13:58:08.929 INFO:teuthology.orchestra.run.smithi017.stdout:(6/11): libatomic-8.5.0-18.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2023-03-29T13:58:08.954 INFO:teuthology.orchestra.run.smithi017.stdout:(7/11): libubsan-8.5.0-18.el8.x86_64.rpm 5.9 MB/s | 152 kB 00:00 2023-03-29T13:58:09.162 INFO:teuthology.orchestra.run.smithi017.stdout:(8/11): libcephsqlite-18.0.0-3104.g6c07bf89.el8 294 kB/s | 135 kB 00:00 2023-03-29T13:58:09.330 INFO:teuthology.orchestra.run.smithi017.stdout:(9/11): yaml-cpp-0.6.2-3.el8.x86_64.rpm 1.0 MB/s | 171 kB 00:00 2023-03-29T13:58:09.480 INFO:teuthology.orchestra.run.smithi017.stdout:(10/11): cryptopp-8.6.0-1.el8.x86_64.rpm 2.4 MB/s | 1.3 MB 00:00 2023-03-29T13:58:12.801 INFO:teuthology.orchestra.run.smithi089.stdout:(11/11): ceph-test-18.0.0-3104.g6c07bf89.el8.x8 20 MB/s | 142 MB 00:07 2023-03-29T13:58:12.804 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:58:12.804 INFO:teuthology.orchestra.run.smithi089.stdout:Total 21 MB/s | 145 MB 00:07 2023-03-29T13:58:12.876 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T13:58:12.898 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T13:58:12.899 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T13:58:13.584 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T13:58:13.587 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T13:58:14.574 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T13:58:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : yaml-cpp-0.6.2-3.el8.x86_64 1/11 2023-03-29T13:58:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cryptopp-8.6.0-1.el8.x86_64 2/11 2023-03-29T13:58:15.205 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libubsan-8.5.0-18.el8.x86_64 3/11 2023-03-29T13:58:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libubsan-8.5.0-18.el8.x86_64 3/11 2023-03-29T13:58:15.355 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libatomic-8.5.0-18.el8.x86_64 4/11 2023-03-29T13:58:15.431 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libatomic-8.5.0-18.el8.x86_64 4/11 2023-03-29T13:58:15.625 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 5/11 2023-03-29T13:58:15.783 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 6/11 2023-03-29T13:58:15.955 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 7/11 2023-03-29T13:58:16.032 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 7/11 2023-03-29T13:58:16.229 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : jq-1.6-6.el8.x86_64 8/11 2023-03-29T13:58:16.399 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-03-29T13:58:16.471 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-03-29T13:58:16.613 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 10/11 2023-03-29T13:58:18.479 INFO:teuthology.orchestra.run.smithi017.stdout:(11/11): ceph-test-18.0.0-3104.g6c07bf89.el8.x8 15 MB/s | 142 MB 00:09 2023-03-29T13:58:18.482 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:58:18.482 INFO:teuthology.orchestra.run.smithi017.stdout:Total 15 MB/s | 145 MB 00:09 2023-03-29T13:58:18.559 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T13:58:18.584 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T13:58:18.584 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T13:58:19.291 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T13:58:19.295 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T13:58:20.323 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T13:58:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : yaml-cpp-0.6.2-3.el8.x86_64 1/11 2023-03-29T13:58:20.800 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cryptopp-8.6.0-1.el8.x86_64 2/11 2023-03-29T13:58:20.920 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libubsan-8.5.0-18.el8.x86_64 3/11 2023-03-29T13:58:20.984 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libubsan-8.5.0-18.el8.x86_64 3/11 2023-03-29T13:58:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libatomic-8.5.0-18.el8.x86_64 4/11 2023-03-29T13:58:21.160 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libatomic-8.5.0-18.el8.x86_64 4/11 2023-03-29T13:58:21.359 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 5/11 2023-03-29T13:58:21.525 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 6/11 2023-03-29T13:58:21.687 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 7/11 2023-03-29T13:58:21.770 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 7/11 2023-03-29T13:58:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : jq-1.6-6.el8.x86_64 8/11 2023-03-29T13:58:22.160 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-03-29T13:58:22.233 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-03-29T13:58:22.370 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 10/11 2023-03-29T13:58:38.966 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 10/11 2023-03-29T13:58:39.223 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : jq-1.6-6.el8.x86_64 3/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libasan6-11.2.1-9.1.el8.x86_64 4/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 6/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 7/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libatomic-8.5.0-18.el8.x86_64 8/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libubsan-8.5.0-18.el8.x86_64 9/11 2023-03-29T13:58:39.526 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cryptopp-8.6.0-1.el8.x86_64 10/11 2023-03-29T13:58:40.135 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : yaml-cpp-0.6.2-3.el8.x86_64 11/11 2023-03-29T13:58:40.135 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:40.135 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: cryptopp-8.6.0-1.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: jq-1.6-6.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: libasan6-11.2.1-9.1.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: libatomic-8.5.0-18.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: libubsan-8.5.0-18.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: yaml-cpp-0.6.2-3.el8.x86_64 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:40.136 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T13:58:40.351 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph 2023-03-29T13:58:41.080 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:21 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T13:58:42.256 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 6.4 k 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 2.2 M 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 1.5 M 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 239 k 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 9.0 M 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 158 M 2023-03-29T13:58:42.260 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-29T13:58:42.261 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout:Install 32 Packages 2023-03-29T13:58:42.262 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:58:42.263 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 174 M 2023-03-29T13:58:42.263 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 2.3 G 2023-03-29T13:58:42.263 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T13:58:42.397 INFO:teuthology.orchestra.run.smithi089.stdout:(1/32): ceph-18.0.0-3104.g6c07bf89.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2023-03-29T13:58:42.588 INFO:teuthology.orchestra.run.smithi089.stdout:(2/32): ceph-mgr-18.0.0-3104.g6c07bf89.el8.x86_ 4.6 MB/s | 1.5 MB 00:00 2023-03-29T13:58:42.630 INFO:teuthology.orchestra.run.smithi089.stdout:(3/32): ceph-mds-18.0.0-3104.g6c07bf89.el8.x86_ 6.0 MB/s | 2.2 MB 00:00 2023-03-29T13:58:42.706 INFO:teuthology.orchestra.run.smithi089.stdout:(4/32): ceph-mgr-modules-core-18.0.0-3104.g6c07 3.1 MB/s | 239 kB 00:00 2023-03-29T13:58:42.772 INFO:teuthology.orchestra.run.smithi089.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-03-29T13:58:42.789 INFO:teuthology.orchestra.run.smithi089.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 2.3 MB/s | 39 kB 00:00 2023-03-29T13:58:42.815 INFO:teuthology.orchestra.run.smithi089.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2023-03-29T13:58:42.832 INFO:teuthology.orchestra.run.smithi089.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 3.2 MB/s | 54 kB 00:00 2023-03-29T13:58:42.866 INFO:teuthology.orchestra.run.smithi089.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-03-29T13:58:42.891 INFO:teuthology.orchestra.run.smithi089.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2023-03-29T13:58:42.925 INFO:teuthology.orchestra.run.smithi089.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-03-29T13:58:42.950 INFO:teuthology.orchestra.run.smithi089.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-03-29T13:58:43.083 INFO:teuthology.orchestra.run.smithi089.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 327 kB/s | 44 kB 00:00 2023-03-29T13:58:43.192 INFO:teuthology.orchestra.run.smithi089.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.7 MB/s | 185 kB 00:00 2023-03-29T13:58:43.242 INFO:teuthology.orchestra.run.smithi089.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 3.4 MB/s | 173 kB 00:00 2023-03-29T13:58:43.326 INFO:teuthology.orchestra.run.smithi089.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.5 MB/s | 384 kB 00:00 2023-03-29T13:58:43.369 INFO:teuthology.orchestra.run.smithi089.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-03-29T13:58:43.410 INFO:teuthology.orchestra.run.smithi089.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 423 kB/s | 18 kB 00:00 2023-03-29T13:58:43.461 INFO:teuthology.orchestra.run.smithi089.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 974 kB/s | 49 kB 00:00 2023-03-29T13:58:43.519 INFO:teuthology.orchestra.run.smithi089.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 999 kB/s | 59 kB 00:00 2023-03-29T13:58:43.603 INFO:teuthology.orchestra.run.smithi089.stdout:(21/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-03-29T13:58:43.653 INFO:teuthology.orchestra.run.smithi089.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-03-29T13:58:43.720 INFO:teuthology.orchestra.run.smithi089.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 280 kB/s | 19 kB 00:00 2023-03-29T13:58:43.771 INFO:teuthology.orchestra.run.smithi089.stdout:(24/32): python3-singledispatch-3.4.0.3-18.el8. 478 kB/s | 24 kB 00:00 2023-03-29T13:58:43.896 INFO:teuthology.orchestra.run.smithi089.stdout:(25/32): ceph-mon-18.0.0-3104.g6c07bf89.el8.x86 6.0 MB/s | 9.0 MB 00:01 2023-03-29T13:58:43.921 INFO:teuthology.orchestra.run.smithi089.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 194 kB/s | 29 kB 00:00 2023-03-29T13:58:43.996 INFO:teuthology.orchestra.run.smithi089.stdout:(27/32): python3-waitress-1.2.1-2.el8.1.noarch. 3.1 MB/s | 241 kB 00:00 2023-03-29T13:58:44.022 INFO:teuthology.orchestra.run.smithi089.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 219 kB/s | 27 kB 00:00 2023-03-29T13:58:44.072 INFO:teuthology.orchestra.run.smithi089.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.3 MB/s | 251 kB 00:00 2023-03-29T13:58:44.097 INFO:teuthology.orchestra.run.smithi089.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2023-03-29T13:58:44.122 INFO:teuthology.orchestra.run.smithi089.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 457 kB/s | 23 kB 00:00 2023-03-29T13:58:46.246 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 10/11 2023-03-29T13:58:46.482 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/11 2023-03-29T13:58:46.772 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 11/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : jq-1.6-6.el8.x86_64 3/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libasan6-11.2.1-9.1.el8.x86_64 4/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 6/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 7/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libatomic-8.5.0-18.el8.x86_64 8/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libubsan-8.5.0-18.el8.x86_64 9/11 2023-03-29T13:58:46.773 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cryptopp-8.6.0-1.el8.x86_64 10/11 2023-03-29T13:58:47.386 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : yaml-cpp-0.6.2-3.el8.x86_64 11/11 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout: cryptopp-8.6.0-1.el8.x86_64 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout: jq-1.6-6.el8.x86_64 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout: libasan6-11.2.1-9.1.el8.x86_64 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout: libatomic-8.5.0-18.el8.x86_64 2023-03-29T13:58:47.387 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:58:47.388 INFO:teuthology.orchestra.run.smithi017.stdout: libubsan-8.5.0-18.el8.x86_64 2023-03-29T13:58:47.388 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-29T13:58:47.388 INFO:teuthology.orchestra.run.smithi017.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-29T13:58:47.388 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-29T13:58:47.388 INFO:teuthology.orchestra.run.smithi017.stdout: yaml-cpp-0.6.2-3.el8.x86_64 2023-03-29T13:58:47.388 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:47.388 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T13:58:47.622 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph 2023-03-29T13:58:48.382 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:27 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T13:58:49.699 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout:======================================================================================= 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout:======================================================================================= 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: ceph x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 6.4 k 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 2.2 M 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 1.5 M 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 239 k 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 9.0 M 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 158 M 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-29T13:58:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-29T13:58:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout:======================================================================================= 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout:Install 32 Packages 2023-03-29T13:58:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:58:49.706 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 174 M 2023-03-29T13:58:49.706 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 2.3 G 2023-03-29T13:58:49.706 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T13:58:49.832 INFO:teuthology.orchestra.run.smithi017.stdout:(1/32): ceph-18.0.0-3104.g6c07bf89.el8.x86_64.r 52 kB/s | 6.4 kB 00:00 2023-03-29T13:58:49.847 INFO:teuthology.orchestra.run.smithi089.stdout:(32/32): ceph-osd-18.0.0-3104.g6c07bf89.el8.x86 22 MB/s | 158 MB 00:07 2023-03-29T13:58:49.851 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:58:49.852 INFO:teuthology.orchestra.run.smithi089.stdout:Total 23 MB/s | 174 MB 00:07 2023-03-29T13:58:49.936 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T13:58:49.974 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T13:58:49.974 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T13:58:50.007 INFO:teuthology.orchestra.run.smithi017.stdout:(2/32): ceph-mgr-18.0.0-3104.g6c07bf89.el8.x86_ 5.0 MB/s | 1.5 MB 00:00 2023-03-29T13:58:50.065 INFO:teuthology.orchestra.run.smithi017.stdout:(3/32): ceph-mds-18.0.0-3104.g6c07bf89.el8.x86_ 6.1 MB/s | 2.2 MB 00:00 2023-03-29T13:58:50.141 INFO:teuthology.orchestra.run.smithi017.stdout:(4/32): ceph-mgr-modules-core-18.0.0-3104.g6c07 3.1 MB/s | 239 kB 00:00 2023-03-29T13:58:50.199 INFO:teuthology.orchestra.run.smithi017.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-03-29T13:58:50.225 INFO:teuthology.orchestra.run.smithi017.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-03-29T13:58:50.242 INFO:teuthology.orchestra.run.smithi017.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 6.0 MB/s | 103 kB 00:00 2023-03-29T13:58:50.267 INFO:teuthology.orchestra.run.smithi017.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-03-29T13:58:50.301 INFO:teuthology.orchestra.run.smithi017.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-03-29T13:58:50.326 INFO:teuthology.orchestra.run.smithi017.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.4 MB/s | 237 kB 00:00 2023-03-29T13:58:50.368 INFO:teuthology.orchestra.run.smithi017.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 13 MB/s | 559 kB 00:00 2023-03-29T13:58:50.393 INFO:teuthology.orchestra.run.smithi017.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-03-29T13:58:50.535 INFO:teuthology.orchestra.run.smithi017.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 308 kB/s | 44 kB 00:00 2023-03-29T13:58:50.676 INFO:teuthology.orchestra.run.smithi017.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.3 MB/s | 185 kB 00:00 2023-03-29T13:58:50.736 INFO:teuthology.orchestra.run.smithi017.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.8 MB/s | 173 kB 00:00 2023-03-29T13:58:50.819 INFO:teuthology.orchestra.run.smithi017.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.5 MB/s | 384 kB 00:00 2023-03-29T13:58:50.870 INFO:teuthology.orchestra.run.smithi017.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-03-29T13:58:50.920 INFO:teuthology.orchestra.run.smithi017.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2023-03-29T13:58:50.970 INFO:teuthology.orchestra.run.smithi017.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 977 kB/s | 49 kB 00:00 2023-03-29T13:58:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2023-03-29T13:58:51.104 INFO:teuthology.orchestra.run.smithi017.stdout:(21/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-03-29T13:58:51.154 INFO:teuthology.orchestra.run.smithi017.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 313 kB/s | 16 kB 00:00 2023-03-29T13:58:51.205 INFO:teuthology.orchestra.run.smithi017.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 375 kB/s | 19 kB 00:00 2023-03-29T13:58:51.255 INFO:teuthology.orchestra.run.smithi017.stdout:(24/32): python3-singledispatch-3.4.0.3-18.el8. 478 kB/s | 24 kB 00:00 2023-03-29T13:58:51.305 INFO:teuthology.orchestra.run.smithi017.stdout:(25/32): python3-tempora-1.14.1-5.el8.noarch.rp 584 kB/s | 29 kB 00:00 2023-03-29T13:58:51.406 INFO:teuthology.orchestra.run.smithi017.stdout:(26/32): ceph-mon-18.0.0-3104.g6c07bf89.el8.x86 5.7 MB/s | 9.0 MB 00:01 2023-03-29T13:58:51.431 INFO:teuthology.orchestra.run.smithi017.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 220 kB/s | 27 kB 00:00 2023-03-29T13:58:51.515 INFO:teuthology.orchestra.run.smithi017.stdout:(28/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.9 MB/s | 251 kB 00:00 2023-03-29T13:58:51.565 INFO:teuthology.orchestra.run.smithi017.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2023-03-29T13:58:51.599 INFO:teuthology.orchestra.run.smithi017.stdout:(30/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.2 MB/s | 241 kB 00:00 2023-03-29T13:58:51.624 INFO:teuthology.orchestra.run.smithi017.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 392 kB/s | 23 kB 00:00 2023-03-29T13:58:52.003 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T13:58:52.007 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T13:58:57.332 INFO:teuthology.orchestra.run.smithi017.stdout:(32/32): ceph-osd-18.0.0-3104.g6c07bf89.el8.x86 22 MB/s | 158 MB 00:07 2023-03-29T13:58:57.336 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:58:57.336 INFO:teuthology.orchestra.run.smithi017.stdout:Total 23 MB/s | 174 MB 00:07 2023-03-29T13:58:57.423 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T13:58:57.466 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T13:58:57.466 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T13:58:58.411 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T13:58:58.415 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T13:59:04.546 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T13:59:04.769 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-03-29T13:59:05.030 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-03-29T13:59:05.244 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-03-29T13:59:05.479 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-03-29T13:59:05.627 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-03-29T13:59:05.790 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-03-29T13:59:06.000 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-03-29T13:59:06.153 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-03-29T13:59:06.351 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-03-29T13:59:06.589 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-03-29T13:59:06.818 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-03-29T13:59:07.067 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-03-29T13:59:07.438 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-03-29T13:59:07.717 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-03-29T13:59:07.905 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-03-29T13:59:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-03-29T13:59:08.390 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-29T13:59:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-03-29T13:59:08.919 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-03-29T13:59:09.246 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-03-29T13:59:09.319 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T13:59:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-03-29T13:59:09.731 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-03-29T13:59:09.959 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-03-29T13:59:10.216 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-03-29T13:59:10.480 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-03-29T13:59:10.582 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-03-29T13:59:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-03-29T13:59:10.879 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-03-29T13:59:10.953 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-03-29T13:59:11.097 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-03-29T13:59:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-03-29T13:59:11.373 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3104.g6c07bf89.el8. 27/32 2023-03-29T13:59:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-03-29T13:59:11.568 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-03-29T13:59:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 28/32 2023-03-29T13:59:11.729 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-03-29T13:59:11.933 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-03-29T13:59:12.203 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-03-29T13:59:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-03-29T13:59:12.632 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-03-29T13:59:13.043 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-03-29T13:59:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-03-29T13:59:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-03-29T13:59:13.760 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-03-29T13:59:13.974 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-29T13:59:14.264 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-03-29T13:59:14.462 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-03-29T13:59:14.681 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-03-29T13:59:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-03-29T13:59:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-03-29T13:59:15.418 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-03-29T13:59:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-03-29T13:59:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-03-29T13:59:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-03-29T13:59:16.832 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3104.g6c07bf89.el8. 27/32 2023-03-29T13:59:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 28/32 2023-03-29T13:59:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 28/32 2023-03-29T13:59:40.156 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 29/32 2023-03-29T13:59:41.075 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 29/32 2023-03-29T13:59:41.301 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 30/32 2023-03-29T13:59:41.518 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 30/32 2023-03-29T13:59:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 31/32 2023-03-29T13:59:41.701 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 31/32 2023-03-29T13:59:41.898 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 32/32 2023-03-29T13:59:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 32/32 2023-03-29T13:59:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 4/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3104.g6c07bf89.el8. 6/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-29T13:59:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-03-29T13:59:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-03-29T13:59:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-03-29T13:59:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-29T13:59:43.258 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-2.6-1.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-29T13:59:43.259 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-29T13:59:43.260 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-29T13:59:43.260 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-29T13:59:43.260 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-29T13:59:43.260 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-29T13:59:43.260 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:59:43.260 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T13:59:43.483 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-base 2023-03-29T13:59:44.221 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:24 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T13:59:45.318 INFO:teuthology.orchestra.run.smithi089.stdout:Package ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T13:59:45.430 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T13:59:45.431 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T13:59:45.432 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T13:59:45.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install cephadm 2023-03-29T13:59:46.499 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:26 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T13:59:47.051 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 28/32 2023-03-29T13:59:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 29/32 2023-03-29T13:59:47.689 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 83 k 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T13:59:47.690 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T13:59:47.691 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 83 k 2023-03-29T13:59:47.691 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 378 k 2023-03-29T13:59:47.691 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T13:59:48.151 INFO:teuthology.orchestra.run.smithi089.stdout:cephadm-18.0.0-3104.g6c07bf89.el8.noarch.rpm 180 kB/s | 83 kB 00:00 2023-03-29T13:59:48.152 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:59:48.152 INFO:teuthology.orchestra.run.smithi089.stdout:Total 180 kB/s | 83 kB 00:00 2023-03-29T13:59:48.153 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T13:59:48.162 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T13:59:48.162 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T13:59:48.165 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T13:59:48.169 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T13:59:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 29/32 2023-03-29T13:59:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 30/32 2023-03-29T13:59:48.656 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 30/32 2023-03-29T13:59:48.717 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T13:59:48.818 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 31/32 2023-03-29T13:59:48.836 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 31/32 2023-03-29T13:59:48.983 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 32/32 2023-03-29T13:59:49.453 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 32/32 2023-03-29T13:59:49.453 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/32 2023-03-29T13:59:49.453 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/32 2023-03-29T13:59:49.453 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 3/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 4/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 5/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3104.g6c07bf89.el8. 6/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-03-29T13:59:49.454 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-03-29T13:59:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-03-29T13:59:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-03-29T13:59:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-03-29T13:59:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-03-29T13:59:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-03-29T13:59:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-03-29T13:59:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-03-29T13:59:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-03-29T13:59:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-03-29T13:59:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-03-29T13:59:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T13:59:50.451 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend-2.6-1.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-29T13:59:50.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:59:50.453 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T13:59:50.675 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-base 2023-03-29T13:59:51.434 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:02:30 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T13:59:52.615 INFO:teuthology.orchestra.run.smithi017.stdout:Package ceph-base-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T13:59:52.736 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T13:59:52.738 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T13:59:52.738 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T13:59:53.062 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install cephadm 2023-03-29T13:59:53.832 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:02:32 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T13:59:55.137 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T13:59:55.138 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 83 k 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T13:59:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T13:59:55.140 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 83 k 2023-03-29T13:59:55.140 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 378 k 2023-03-29T13:59:55.140 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T13:59:55.559 INFO:teuthology.orchestra.run.smithi017.stdout:cephadm-18.0.0-3104.g6c07bf89.el8.noarch.rpm 197 kB/s | 83 kB 00:00 2023-03-29T13:59:55.560 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T13:59:55.560 INFO:teuthology.orchestra.run.smithi017.stdout:Total 197 kB/s | 83 kB 00:00 2023-03-29T13:59:55.561 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T13:59:55.570 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T13:59:55.570 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T13:59:55.573 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T13:59:55.577 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T13:59:56.073 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T13:59:58.479 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:01.674 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:01.674 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:01.675 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:01.675 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:01.675 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:01.675 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:01.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-immutable-object-cache 2023-03-29T14:00:02.621 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:42 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:03.826 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:03.827 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:03.827 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2023-03-29T14:00:03.827 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:03.827 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 157 k 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 157 k 2023-03-29T14:00:03.828 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 540 k 2023-03-29T14:00:03.829 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:04.024 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-immutable-object-cache-18.0.0-3104.g6c07bf 804 kB/s | 157 kB 00:00 2023-03-29T14:00:04.024 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:04.025 INFO:teuthology.orchestra.run.smithi089.stdout:Total 799 kB/s | 157 kB 00:00 2023-03-29T14:00:04.025 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:04.035 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:04.035 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:04.084 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:04.087 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:04.204 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89. 1/1 2023-03-29T14:00:04.830 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89. 1/1 2023-03-29T14:00:05.302 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:05.344 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89. 1/1 2023-03-29T14:00:05.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:05.344 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:05.344 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:00:05.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:05.344 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:05.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr 2023-03-29T14:00:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:06.253 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:46 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:06.729 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:06.729 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:06.920 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-immutable-object-cache 2023-03-29T14:00:07.343 INFO:teuthology.orchestra.run.smithi089.stdout:Package ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:00:07.453 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:07.454 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:00:07.454 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:07.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-dashboard 2023-03-29T14:00:07.686 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:02:46 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:08.267 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:48 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:08.977 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:08.977 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 157 k 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 157 k 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 540 k 2023-03-29T14:00:08.978 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:09.188 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-immutable-object-cache-18.0.0-3104.g6c07bf 749 kB/s | 157 kB 00:00 2023-03-29T14:00:09.189 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:09.189 INFO:teuthology.orchestra.run.smithi017.stdout:Total 744 kB/s | 157 kB 00:00 2023-03-29T14:00:09.189 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:09.201 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:09.201 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:09.256 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:09.259 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:09.378 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:09.464 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:09.465 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:09.465 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T14:00:09.466 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:09.466 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:09.466 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 1.7 M 2023-03-29T14:00:09.466 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T14:00:09.466 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 23 k 2023-03-29T14:00:09.466 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 15 k 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout:Install 6 Packages 2023-03-29T14:00:09.467 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:09.468 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 2.0 M 2023-03-29T14:00:09.468 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 8.1 M 2023-03-29T14:00:09.468 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:09.549 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89. 1/1 2023-03-29T14:00:09.602 INFO:teuthology.orchestra.run.smithi089.stdout:(1/6): ceph-grafana-dashboards-18.0.0-3104.g6c0 172 kB/s | 23 kB 00:00 2023-03-29T14:00:09.627 INFO:teuthology.orchestra.run.smithi089.stdout:(2/6): ceph-prometheus-alerts-18.0.0-3104.g6c07 91 kB/s | 15 kB 00:00 2023-03-29T14:00:09.686 INFO:teuthology.orchestra.run.smithi089.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 509 kB/s | 43 kB 00:00 2023-03-29T14:00:09.736 INFO:teuthology.orchestra.run.smithi089.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 309 kB/s | 34 kB 00:00 2023-03-29T14:00:09.778 INFO:teuthology.orchestra.run.smithi089.stdout:(5/6): ceph-mgr-dashboard-18.0.0-3104.g6c07bf89 5.5 MB/s | 1.7 MB 00:00 2023-03-29T14:00:09.869 INFO:teuthology.orchestra.run.smithi089.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.0 MB/s | 196 kB 00:00 2023-03-29T14:00:09.872 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:09.872 INFO:teuthology.orchestra.run.smithi089.stdout:Total 4.9 MB/s | 2.0 MB 00:00 2023-03-29T14:00:09.882 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:09.891 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:09.891 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:09.973 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:09.977 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:10.005 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89. 1/1 2023-03-29T14:00:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:10.493 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-03-29T14:00:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89. 1/1 2023-03-29T14:00:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:10.531 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:00:10.531 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:10.531 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:10.705 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-03-29T14:00:10.706 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr 2023-03-29T14:00:10.907 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-03-29T14:00:11.035 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3104.g6c07bf89.el8.n 4/6 2023-03-29T14:00:11.426 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3104.g6c07bf89.el8. 5/6 2023-03-29T14:00:11.489 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:02:50 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarc 6/6 2023-03-29T14:00:11.987 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarc 6/6 2023-03-29T14:00:11.987 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3104.g6c07bf89.el8. 1/6 2023-03-29T14:00:11.987 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarc 2/6 2023-03-29T14:00:11.987 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3104.g6c07bf89.el8.n 3/6 2023-03-29T14:00:11.987 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-03-29T14:00:11.987 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-03-29T14:00:12.655 INFO:teuthology.orchestra.run.smithi017.stdout:Package ceph-mgr-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:00:12.778 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:12.779 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:00:12.779 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:12.864 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-dashboard 2023-03-29T14:00:13.276 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-03-29T14:00:13.276 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:13.276 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:13.276 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:13.276 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:13.276 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:13.276 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-29T14:00:13.277 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-29T14:00:13.277 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-29T14:00:13.277 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:13.277 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:13.403 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-diskprediction-local 2023-03-29T14:00:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:02:52 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:14.132 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:54 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:14.919 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:14.920 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T14:00:14.920 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:14.920 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 1.7 M 2023-03-29T14:00:14.920 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T14:00:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 23 k 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 15 k 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout:Install 6 Packages 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:14.921 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 2.0 M 2023-03-29T14:00:14.922 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 8.1 M 2023-03-29T14:00:14.922 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:15.065 INFO:teuthology.orchestra.run.smithi017.stdout:(1/6): ceph-grafana-dashboards-18.0.0-3104.g6c0 162 kB/s | 23 kB 00:00 2023-03-29T14:00:15.091 INFO:teuthology.orchestra.run.smithi017.stdout:(2/6): ceph-prometheus-alerts-18.0.0-3104.g6c07 87 kB/s | 15 kB 00:00 2023-03-29T14:00:15.158 INFO:teuthology.orchestra.run.smithi017.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 463 kB/s | 43 kB 00:00 2023-03-29T14:00:15.200 INFO:teuthology.orchestra.run.smithi017.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 310 kB/s | 34 kB 00:00 2023-03-29T14:00:15.241 INFO:teuthology.orchestra.run.smithi017.stdout:(5/6): ceph-mgr-dashboard-18.0.0-3104.g6c07bf89 5.3 MB/s | 1.7 MB 00:00 2023-03-29T14:00:15.335 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:15.336 INFO:teuthology.orchestra.run.smithi089.stdout:===================================================================================================== 2023-03-29T14:00:15.336 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T14:00:15.336 INFO:teuthology.orchestra.run.smithi089.stdout:===================================================================================================== 2023-03-29T14:00:15.336 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:15.336 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 7.4 M 2023-03-29T14:00:15.336 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout:===================================================================================================== 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout:Install 8 Packages 2023-03-29T14:00:15.337 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:15.338 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 35 M 2023-03-29T14:00:15.338 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 210 M 2023-03-29T14:00:15.338 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:15.350 INFO:teuthology.orchestra.run.smithi017.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.0 MB/s | 196 kB 00:00 2023-03-29T14:00:15.352 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:15.353 INFO:teuthology.orchestra.run.smithi017.stdout:Total 4.6 MB/s | 2.0 MB 00:00 2023-03-29T14:00:15.364 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:15.373 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:15.374 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:15.463 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:15.466 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:(1/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 16 MB/s | 4.9 MB 00:00 2023-03-29T14:00:15.719 INFO:teuthology.orchestra.run.smithi089.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-03-29T14:00:15.749 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:15.752 INFO:teuthology.orchestra.run.smithi089.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2023-03-29T14:00:15.844 INFO:teuthology.orchestra.run.smithi089.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2023-03-29T14:00:15.936 INFO:teuthology.orchestra.run.smithi089.stdout:(5/8): ceph-mgr-diskprediction-local-18.0.0-310 12 MB/s | 7.4 MB 00:00 2023-03-29T14:00:15.962 INFO:teuthology.orchestra.run.smithi089.stdout:(6/8): libgfortran-8.5.0-18.el8.x86_64.rpm 5.4 MB/s | 645 kB 00:00 2023-03-29T14:00:15.998 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-03-29T14:00:16.054 INFO:teuthology.orchestra.run.smithi089.stdout:(7/8): libquadmath-8.5.0-18.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2023-03-29T14:00:16.237 INFO:teuthology.orchestra.run.smithi089.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 28 MB/s | 14 MB 00:00 2023-03-29T14:00:16.240 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:16.240 INFO:teuthology.orchestra.run.smithi089.stdout:Total 39 MB/s | 35 MB 00:00 2023-03-29T14:00:16.250 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-03-29T14:00:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-03-29T14:00:16.471 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:16.489 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:16.489 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:16.580 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3104.g6c07bf89.el8.n 4/6 2023-03-29T14:00:16.871 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:16.875 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:16.979 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3104.g6c07bf89.el8. 5/6 2023-03-29T14:00:17.229 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarc 6/6 2023-03-29T14:00:17.433 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:17.549 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarc 6/6 2023-03-29T14:00:17.549 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3104.g6c07bf89.el8. 1/6 2023-03-29T14:00:17.549 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarc 2/6 2023-03-29T14:00:17.549 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3104.g6c07bf89.el8.n 3/6 2023-03-29T14:00:17.549 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-03-29T14:00:17.550 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-03-29T14:00:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-29T14:00:17.744 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-29T14:00:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-29T14:00:18.086 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:18.087 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:18.202 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-diskprediction-local 2023-03-29T14:00:18.487 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-29T14:00:18.686 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-03-29T14:00:18.964 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:02:57 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:19.299 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-03-29T14:00:19.461 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-29T14:00:20.140 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-29T14:00:20.259 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout:===================================================================================================== 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout:===================================================================================================== 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 7.4 M 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-29T14:00:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout:===================================================================================================== 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout:Install 8 Packages 2023-03-29T14:00:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:20.263 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 35 M 2023-03-29T14:00:20.263 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 210 M 2023-03-29T14:00:20.263 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:20.440 INFO:teuthology.orchestra.run.smithi017.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 27 MB/s | 4.7 MB 00:00 2023-03-29T14:00:20.489 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-03-29T14:00:20.557 INFO:teuthology.orchestra.run.smithi017.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 32 MB/s | 3.7 MB 00:00 2023-03-29T14:00:20.591 INFO:teuthology.orchestra.run.smithi017.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.7 MB/s | 225 kB 00:00 2023-03-29T14:00:20.683 INFO:teuthology.orchestra.run.smithi017.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.9 MB 00:00 2023-03-29T14:00:20.733 INFO:teuthology.orchestra.run.smithi017.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 13 MB/s | 645 kB 00:00 2023-03-29T14:00:20.761 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-29T14:00:20.825 INFO:teuthology.orchestra.run.smithi017.stdout:(6/8): ceph-mgr-diskprediction-local-18.0.0-310 13 MB/s | 7.4 MB 00:00 2023-03-29T14:00:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:(7/8): libquadmath-8.5.0-18.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2023-03-29T14:00:21.051 INFO:teuthology.orchestra.run.smithi017.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 30 MB/s | 14 MB 00:00 2023-03-29T14:00:21.053 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:21.053 INFO:teuthology.orchestra.run.smithi017.stdout:Total 45 MB/s | 35 MB 00:00 2023-03-29T14:00:21.291 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:21.310 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:21.310 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:21.874 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:21.878 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-29T14:00:22.958 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:23.138 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-29T14:00:23.277 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-29T14:00:23.421 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-29T14:00:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-03-29T14:00:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf8 8/8 2023-03-29T14:00:24.054 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-29T14:00:24.230 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-03-29T14:00:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf8 8/8 2023-03-29T14:00:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf8 1/8 2023-03-29T14:00:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-03-29T14:00:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-03-29T14:00:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-03-29T14:00:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-03-29T14:00:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-03-29T14:00:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-03-29T14:00:24.861 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-03-29T14:00:24.861 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:24.862 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-03-29T14:00:24.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-rook 2023-03-29T14:00:25.030 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-29T14:00:25.716 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:05 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-29T14:00:26.130 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-03-29T14:00:26.346 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-29T14:00:26.917 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================== 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================== 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 47 k 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-29T14:00:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================== 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout:Install 13 Packages 2023-03-29T14:00:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:26.921 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 1.9 M 2023-03-29T14:00:26.921 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 25 M 2023-03-29T14:00:26.921 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:26.994 INFO:teuthology.orchestra.run.smithi089.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 375 kB/s | 27 kB 00:00 2023-03-29T14:00:27.019 INFO:teuthology.orchestra.run.smithi089.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 208 kB/s | 20 kB 00:00 2023-03-29T14:00:27.044 INFO:teuthology.orchestra.run.smithi089.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-03-29T14:00:27.069 INFO:teuthology.orchestra.run.smithi089.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-03-29T14:00:27.095 INFO:teuthology.orchestra.run.smithi089.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-03-29T14:00:27.120 INFO:teuthology.orchestra.run.smithi089.stdout:(6/13): ceph-mgr-rook-18.0.0-3104.g6c07bf89.el8 240 kB/s | 47 kB 00:00 2023-03-29T14:00:27.145 INFO:teuthology.orchestra.run.smithi089.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 562 kB/s | 43 kB 00:00 2023-03-29T14:00:27.237 INFO:teuthology.orchestra.run.smithi089.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 232 kB/s | 33 kB 00:00 2023-03-29T14:00:27.263 INFO:teuthology.orchestra.run.smithi089.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 114 kB/s | 16 kB 00:00 2023-03-29T14:00:27.288 INFO:teuthology.orchestra.run.smithi089.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 571 kB/s | 81 kB 00:00 2023-03-29T14:00:27.347 INFO:teuthology.orchestra.run.smithi089.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 735 kB/s | 61 kB 00:00 2023-03-29T14:00:27.372 INFO:teuthology.orchestra.run.smithi089.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 732 kB/s | 61 kB 00:00 2023-03-29T14:00:27.497 INFO:teuthology.orchestra.run.smithi089.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 4.5 MB/s | 1.2 MB 00:00 2023-03-29T14:00:27.498 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:27.499 INFO:teuthology.orchestra.run.smithi089.stdout:Total 3.3 MB/s | 1.9 MB 00:00 2023-03-29T14:00:27.551 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:27.566 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:27.566 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:27.665 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:27.669 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:28.182 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:28.220 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-29T14:00:28.454 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-03-29T14:00:28.644 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-03-29T14:00:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-03-29T14:00:29.133 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-03-29T14:00:29.349 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-03-29T14:00:29.520 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-03-29T14:00:29.604 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-03-29T14:00:29.713 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf8 8/8 2023-03-29T14:00:29.929 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf8 8/8 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf8 1/8 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-03-29T14:00:30.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-03-29T14:00:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:30.612 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:30.726 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-rook 2023-03-29T14:00:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-03-29T14:00:31.488 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:10 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:31.541 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-03-29T14:00:31.758 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-03-29T14:00:31.977 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-03-29T14:00:32.185 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 13/13 2023-03-29T14:00:32.484 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 13/13 2023-03-29T14:00:32.484 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 1/13 2023-03-29T14:00:32.484 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-03-29T14:00:32.484 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-03-29T14:00:32.484 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-03-29T14:00:32.484 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-03-29T14:00:32.484 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-03-29T14:00:32.485 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-03-29T14:00:32.485 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-03-29T14:00:32.485 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-03-29T14:00:32.485 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-03-29T14:00:32.485 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-03-29T14:00:32.485 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-03-29T14:00:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:======================================================================================== 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:======================================================================================== 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 47 k 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-29T14:00:32.783 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:======================================================================================== 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:Install 13 Packages 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 1.9 M 2023-03-29T14:00:32.785 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 25 M 2023-03-29T14:00:32.785 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:32.847 INFO:teuthology.orchestra.run.smithi017.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 441 kB/s | 27 kB 00:00 2023-03-29T14:00:32.872 INFO:teuthology.orchestra.run.smithi017.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 234 kB/s | 20 kB 00:00 2023-03-29T14:00:32.898 INFO:teuthology.orchestra.run.smithi017.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-03-29T14:00:32.942 INFO:teuthology.orchestra.run.smithi017.stdout:(4/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.4 MB/s | 155 kB 00:00 2023-03-29T14:00:32.965 INFO:teuthology.orchestra.run.smithi017.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.2 MB/s | 110 kB 00:00 2023-03-29T14:00:32.990 INFO:teuthology.orchestra.run.smithi017.stdout:(6/13): ceph-mgr-rook-18.0.0-3104.g6c07bf89.el8 233 kB/s | 47 kB 00:00 2023-03-29T14:00:33.016 INFO:teuthology.orchestra.run.smithi017.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 578 kB/s | 43 kB 00:00 2023-03-29T14:00:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:33.054 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:33.133 INFO:teuthology.orchestra.run.smithi017.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 115 kB/s | 16 kB 00:00 2023-03-29T14:00:33.158 INFO:teuthology.orchestra.run.smithi017.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 171 kB/s | 33 kB 00:00 2023-03-29T14:00:33.162 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-cephadm 2023-03-29T14:00:33.208 INFO:teuthology.orchestra.run.smithi017.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 423 kB/s | 81 kB 00:00 2023-03-29T14:00:33.267 INFO:teuthology.orchestra.run.smithi017.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 566 kB/s | 61 kB 00:00 2023-03-29T14:00:33.284 INFO:teuthology.orchestra.run.smithi017.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 814 kB/s | 61 kB 00:00 2023-03-29T14:00:33.467 INFO:teuthology.orchestra.run.smithi017.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.5 MB/s | 1.2 MB 00:00 2023-03-29T14:00:33.469 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:33.469 INFO:teuthology.orchestra.run.smithi017.stdout:Total 2.8 MB/s | 1.9 MB 00:00 2023-03-29T14:00:33.527 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:33.544 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:33.544 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:33.650 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:33.653 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:33.881 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:13 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:34.155 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:34.409 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-03-29T14:00:34.632 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-03-29T14:00:34.883 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-03-29T14:00:35.093 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 123 k 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-29T14:00:35.094 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:Install 5 Packages 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 6.0 M 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 26 M 2023-03-29T14:00:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:35.111 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-03-29T14:00:35.187 INFO:teuthology.orchestra.run.smithi089.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.9 MB/s | 538 kB 00:00 2023-03-29T14:00:35.307 INFO:teuthology.orchestra.run.smithi089.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-03-29T14:00:35.320 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-03-29T14:00:35.333 INFO:teuthology.orchestra.run.smithi089.stdout:(3/5): ceph-mgr-cephadm-18.0.0-3104.g6c07bf89.e 523 kB/s | 123 kB 00:00 2023-03-29T14:00:35.449 INFO:teuthology.orchestra.run.smithi089.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 425 kB/s | 60 kB 00:00 2023-03-29T14:00:35.491 INFO:teuthology.orchestra.run.smithi089.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.5 MB/s | 476 kB 00:00 2023-03-29T14:00:35.493 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:35.493 INFO:teuthology.orchestra.run.smithi089.stdout:Total 15 MB/s | 6.0 MB 00:00 2023-03-29T14:00:35.544 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:35.555 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:35.555 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:35.562 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-03-29T14:00:35.686 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:35.689 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:35.927 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-03-29T14:00:35.968 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:36.145 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-03-29T14:00:36.339 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-03-29T14:00:37.261 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-03-29T14:00:37.286 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-03-29T14:00:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-03-29T14:00:37.670 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-03-29T14:00:37.727 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-03-29T14:00:37.922 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-03-29T14:00:37.930 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-03-29T14:00:38.209 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 5/5 2023-03-29T14:00:38.337 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 13/13 2023-03-29T14:00:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 5/5 2023-03-29T14:00:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/5 2023-03-29T14:00:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-03-29T14:00:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-03-29T14:00:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 13/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 1/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-03-29T14:00:38.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:39.012 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:39.130 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-fuse 2023-03-29T14:00:39.267 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-03-29T14:00:39.267 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:39.268 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:39.382 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-cephadm 2023-03-29T14:00:39.855 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:19 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:40.142 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:19 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:41.049 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:41.050 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:41.050 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T14:00:41.050 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:41.050 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:41.050 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 815 k 2023-03-29T14:00:41.050 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-03-29T14:00:41.050 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout:Install 2 Packages 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 898 k 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 3.3 M 2023-03-29T14:00:41.051 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:41.125 INFO:teuthology.orchestra.run.smithi089.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-03-29T14:00:41.317 INFO:teuthology.orchestra.run.smithi089.stdout:(2/2): ceph-fuse-18.0.0-3104.g6c07bf89.el8.x86_ 3.0 MB/s | 815 kB 00:00 2023-03-29T14:00:41.318 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:41.318 INFO:teuthology.orchestra.run.smithi089.stdout:Total 3.3 MB/s | 898 kB 00:00 2023-03-29T14:00:41.329 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:41.350 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:41.350 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:41.441 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:41.445 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:41.453 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:41.454 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:41.454 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T14:00:41.454 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:41.454 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:41.454 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 123 k 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout:Install 5 Packages 2023-03-29T14:00:41.455 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:41.456 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 6.0 M 2023-03-29T14:00:41.456 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 26 M 2023-03-29T14:00:41.456 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:41.550 INFO:teuthology.orchestra.run.smithi017.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.7 MB/s | 538 kB 00:00 2023-03-29T14:00:41.624 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:41.650 INFO:teuthology.orchestra.run.smithi017.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-03-29T14:00:41.692 INFO:teuthology.orchestra.run.smithi017.stdout:(3/5): ceph-mgr-cephadm-18.0.0-3104.g6c07bf89.e 525 kB/s | 123 kB 00:00 2023-03-29T14:00:41.776 INFO:teuthology.orchestra.run.smithi017.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.1 MB/s | 476 kB 00:00 2023-03-29T14:00:41.792 INFO:teuthology.orchestra.run.smithi017.stdout:(5/5): python3-natsort-7.1.1-2.el8.noarch.rpm 424 kB/s | 60 kB 00:00 2023-03-29T14:00:41.794 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:41.794 INFO:teuthology.orchestra.run.smithi017.stdout:Total 18 MB/s | 6.0 MB 00:00 2023-03-29T14:00:41.847 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:41.859 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:41.859 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:41.917 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-03-29T14:00:42.002 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:42.005 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:42.140 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/2 2023-03-29T14:00:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:42.524 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/2 2023-03-29T14:00:42.525 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/2 2023-03-29T14:00:42.601 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-03-29T14:00:43.057 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-03-29T14:00:43.058 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:43.058 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-03-29T14:00:43.058 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:43.058 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:43.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-volume 2023-03-29T14:00:43.598 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-03-29T14:00:44.001 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:24 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:44.329 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-03-29T14:00:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-03-29T14:00:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 5/5 2023-03-29T14:00:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 5/5 2023-03-29T14:00:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 1/5 2023-03-29T14:00:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-03-29T14:00:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-03-29T14:00:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-03-29T14:00:45.212 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:45.212 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 263 k 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T14:00:45.213 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:45.214 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 263 k 2023-03-29T14:00:45.214 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 1.3 M 2023-03-29T14:00:45.214 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:45.413 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-volume-18.0.0-3104.g6c07bf89.el8.noarch.rp 1.3 MB/s | 263 kB 00:00 2023-03-29T14:00:45.414 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:45.414 INFO:teuthology.orchestra.run.smithi089.stdout:Total 1.3 MB/s | 263 kB 00:00 2023-03-29T14:00:45.414 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:45.424 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:45.425 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:45.523 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:45.526 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:45.683 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:45.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-fuse 2023-03-29T14:00:45.974 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:46.398 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:46.560 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:25 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:46.919 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:46.919 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:46.919 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:46.919 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:46.919 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:46.919 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:47.113 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install librados-devel 2023-03-29T14:00:47.854 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:27 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:47.856 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:47.856 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T14:00:47.857 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:47.857 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 815 k 2023-03-29T14:00:47.858 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-03-29T14:00:47.858 INFO:teuthology.orchestra.run.smithi017.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-29T14:00:47.858 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:47.858 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:47.858 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:47.858 INFO:teuthology.orchestra.run.smithi017.stdout:Install 2 Packages 2023-03-29T14:00:47.858 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:47.859 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 898 k 2023-03-29T14:00:47.859 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 3.3 M 2023-03-29T14:00:47.859 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:47.921 INFO:teuthology.orchestra.run.smithi017.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.3 MB/s | 83 kB 00:00 2023-03-29T14:00:48.121 INFO:teuthology.orchestra.run.smithi017.stdout:(2/2): ceph-fuse-18.0.0-3104.g6c07bf89.el8.x86_ 3.0 MB/s | 815 kB 00:00 2023-03-29T14:00:48.122 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:48.122 INFO:teuthology.orchestra.run.smithi017.stdout:Total 3.3 MB/s | 898 kB 00:00 2023-03-29T14:00:48.133 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:48.155 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:48.155 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:48.257 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:48.260 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:48.451 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:48.731 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-03-29T14:00:48.952 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/2 2023-03-29T14:00:49.055 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:49.055 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2023-03-29T14:00:49.056 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:49.056 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 139 k 2023-03-29T14:00:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:49.056 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:49.056 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:49.057 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T14:00:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:49.057 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 139 k 2023-03-29T14:00:49.058 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 512 k 2023-03-29T14:00:49.058 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:49.252 INFO:teuthology.orchestra.run.smithi089.stdout:librados-devel-18.0.0-3104.g6c07bf89.el8.x86_64 711 kB/s | 139 kB 00:00 2023-03-29T14:00:49.253 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:49.253 INFO:teuthology.orchestra.run.smithi089.stdout:Total 707 kB/s | 139 kB 00:00 2023-03-29T14:00:49.253 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:49.258 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:49.258 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:49.311 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:49.315 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:49.363 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2/2 2023-03-29T14:00:49.363 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/2 2023-03-29T14:00:49.447 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:49.654 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:49.951 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-03-29T14:00:49.951 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:49.951 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:49.952 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-03-29T14:00:49.952 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:49.952 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:50.152 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-volume 2023-03-29T14:00:50.918 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:29 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:51.134 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:51.135 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:51.135 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:51.135 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:00:51.135 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:51.135 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:51.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install libcephfs2 2023-03-29T14:00:52.091 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:32 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:52.216 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:52.216 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume noarch 2:18.0.0-3104.g6c07bf89.el8 ceph-noarch 263 k 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:52.217 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 263 k 2023-03-29T14:00:52.218 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 1.3 M 2023-03-29T14:00:52.218 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:52.425 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-volume-18.0.0-3104.g6c07bf89.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-03-29T14:00:52.426 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:52.426 INFO:teuthology.orchestra.run.smithi017.stdout:Total 1.2 MB/s | 263 kB 00:00 2023-03-29T14:00:52.426 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:52.437 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:52.437 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:52.531 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:52.535 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:53.002 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:53.195 INFO:teuthology.orchestra.run.smithi089.stdout:Package libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:00:53.305 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:00:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:53.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install libcephfs-devel 2023-03-29T14:00:53.486 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:54.019 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 1/1 2023-03-29T14:00:54.019 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:54.019 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:54.019 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume-2:18.0.0-3104.g6c07bf89.el8.noarch 2023-03-29T14:00:54.019 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:54.019 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:54.120 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:34 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:54.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install librados-devel 2023-03-29T14:00:54.997 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:33 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:55.319 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:55.320 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:55.320 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2023-03-29T14:00:55.320 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:55.320 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:00:55.320 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 30 k 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 30 k 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 137 k 2023-03-29T14:00:55.321 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:00:55.481 INFO:teuthology.orchestra.run.smithi089.stdout:libcephfs-devel-18.0.0-3104.g6c07bf89.el8.x86_6 189 kB/s | 30 kB 00:00 2023-03-29T14:00:55.482 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:55.482 INFO:teuthology.orchestra.run.smithi089.stdout:Total 188 kB/s | 30 kB 00:00 2023-03-29T14:00:55.482 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:00:55.485 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:00:55.486 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:00:55.520 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:00:55.524 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:00:55.583 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:00:55.811 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:56.087 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:56.299 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:00:56.300 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:56.300 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2023-03-29T14:00:56.300 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:56.300 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:00:56.300 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 139 k 2023-03-29T14:00:56.300 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:56.301 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:00:56.301 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:00:56.301 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T14:00:56.301 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:56.301 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 139 k 2023-03-29T14:00:56.301 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 512 k 2023-03-29T14:00:56.301 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:00:56.827 INFO:teuthology.orchestra.run.smithi017.stdout:librados-devel-18.0.0-3104.g6c07bf89.el8.x86_64 263 kB/s | 139 kB 00:00 2023-03-29T14:00:56.828 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:00:56.828 INFO:teuthology.orchestra.run.smithi017.stdout:Total 262 kB/s | 139 kB 00:00 2023-03-29T14:00:56.828 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:00:56.835 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:00:56.835 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:00:56.895 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:00:56.899 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:00:57.029 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:00:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:57.344 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:57.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:57.344 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:00:57.344 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:00:57.345 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:00:57.345 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:57.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install librados2 2023-03-29T14:00:57.527 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:58.058 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:00:58.059 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:58.059 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:00:58.059 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:00:58.059 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:00:58.059 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:00:58.174 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:38 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:00:58.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install libcephfs2 2023-03-29T14:00:59.023 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:38 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:00:59.270 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:00:59.379 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:00:59.381 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:00:59.381 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:00:59.470 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install librbd1 2023-03-29T14:01:00.188 INFO:teuthology.orchestra.run.smithi017.stdout:Package libcephfs2-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:00.199 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:40 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:00.310 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:00.311 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:01:00.311 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:00.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install libcephfs-devel 2023-03-29T14:01:01.287 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:01.398 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:01.399 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:01:01.399 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:01.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-rados 2023-03-29T14:01:01.533 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:40 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:02.219 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:42 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:02.839 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 30 k 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T14:01:02.840 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:02.841 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 30 k 2023-03-29T14:01:02.841 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 137 k 2023-03-29T14:01:02.841 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:01:02.984 INFO:teuthology.orchestra.run.smithi017.stdout:libcephfs-devel-18.0.0-3104.g6c07bf89.el8.x86_6 214 kB/s | 30 kB 00:00 2023-03-29T14:01:02.984 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:01:02.984 INFO:teuthology.orchestra.run.smithi017.stdout:Total 212 kB/s | 30 kB 00:00 2023-03-29T14:01:02.985 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:01:02.988 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:01:02.988 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:01:03.006 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:01:03.010 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:01:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:01:03.265 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:03.309 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:03.417 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:03.418 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:01:03.418 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:03.506 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-rgw 2023-03-29T14:01:03.536 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:04.030 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:01:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:01:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:04.030 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:04.143 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install librados2 2023-03-29T14:01:04.234 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:44 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:04.898 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:43 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:05.325 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:05.436 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:05.437 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:01:05.437 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:05.521 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-cephfs 2023-03-29T14:01:06.067 INFO:teuthology.orchestra.run.smithi017.stdout:Package librados2-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:06.188 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:06.189 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:01:06.190 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:06.249 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:46 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:06.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install librbd1 2023-03-29T14:01:07.103 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:46 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:07.345 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:07.458 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:07.459 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:01:07.459 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:07.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-rbd 2023-03-29T14:01:08.268 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:48 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:08.279 INFO:teuthology.orchestra.run.smithi017.stdout:Package librbd1-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:08.403 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:08.404 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:01:08.404 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:08.514 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-rados 2023-03-29T14:01:09.283 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:48 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:09.368 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:09.477 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:09.478 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-03-29T14:01:09.478 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:09.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install rbd-fuse 2023-03-29T14:01:10.287 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:50 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:10.471 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-rados-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:10.596 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:10.597 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:01:10.597 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:10.684 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-rgw 2023-03-29T14:01:11.453 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:50 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:11.493 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 91 k 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T14:01:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:11.495 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 91 k 2023-03-29T14:01:11.495 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 303 k 2023-03-29T14:01:11.495 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:01:11.657 INFO:teuthology.orchestra.run.smithi089.stdout:rbd-fuse-18.0.0-3104.g6c07bf89.el8.x86_64.rpm 562 kB/s | 91 kB 00:00 2023-03-29T14:01:11.658 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:01:11.658 INFO:teuthology.orchestra.run.smithi089.stdout:Total 558 kB/s | 91 kB 00:00 2023-03-29T14:01:11.658 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:01:11.668 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:01:11.668 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:01:11.720 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:01:11.724 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:01:11.836 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:01:12.101 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:12.407 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:12.618 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-rgw-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:12.742 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:12.744 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:01:12.744 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:12.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-cephfs 2023-03-29T14:01:12.905 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:12.906 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:12.906 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:01:12.906 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:01:12.906 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:12.906 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:13.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install rbd-mirror 2023-03-29T14:01:13.598 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:52 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:13.830 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:53 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:14.765 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-cephfs-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:14.888 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:14.890 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:01:14.890 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:14.978 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-rbd 2023-03-29T14:01:15.046 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:15.047 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:15.047 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2023-03-29T14:01:15.047 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:15.047 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:01:15.047 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 3.3 M 2023-03-29T14:01:15.047 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:15.048 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:01:15.048 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:15.048 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T14:01:15.048 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:15.048 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 3.3 M 2023-03-29T14:01:15.048 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 18 M 2023-03-29T14:01:15.048 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:01:15.562 INFO:teuthology.orchestra.run.smithi089.stdout:rbd-mirror-18.0.0-3104.g6c07bf89.el8.x86_64.rpm 6.5 MB/s | 3.3 MB 00:00 2023-03-29T14:01:15.562 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:01:15.562 INFO:teuthology.orchestra.run.smithi089.stdout:Total 6.4 MB/s | 3.3 MB 00:00 2023-03-29T14:01:15.563 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:01:15.574 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:01:15.574 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:01:15.636 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:01:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:01:15.755 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:54 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:16.080 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:01:16.261 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:16.709 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:16.927 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-rbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 is already installed. 2023-03-29T14:01:17.048 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:17.050 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-03-29T14:01:17.050 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:17.192 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:17.192 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:17.192 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:01:17.192 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:01:17.192 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:17.192 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:17.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install rbd-nbd 2023-03-29T14:01:17.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install rbd-fuse 2023-03-29T14:01:18.121 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:03:58 ago on Wed 29 Mar 2023 01:57:20 PM UTC. 2023-03-29T14:01:18.227 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:03:57 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:19.317 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 177 k 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-03-29T14:01:19.318 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-03-29T14:01:19.319 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2023-03-29T14:01:19.319 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:19.319 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 177 k 2023-03-29T14:01:19.319 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 626 k 2023-03-29T14:01:19.320 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-03-29T14:01:19.509 INFO:teuthology.orchestra.run.smithi089.stdout:rbd-nbd-18.0.0-3104.g6c07bf89.el8.x86_64.rpm 932 kB/s | 177 kB 00:00 2023-03-29T14:01:19.510 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:01:19.510 INFO:teuthology.orchestra.run.smithi089.stdout:Total 926 kB/s | 177 kB 00:00 2023-03-29T14:01:19.510 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-03-29T14:01:19.516 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:19.516 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2023-03-29T14:01:19.516 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:19.516 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:01:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 91 k 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 91 k 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 303 k 2023-03-29T14:01:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:01:19.521 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-03-29T14:01:19.521 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-03-29T14:01:19.572 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-03-29T14:01:19.575 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-03-29T14:01:19.690 INFO:teuthology.orchestra.run.smithi017.stdout:rbd-fuse-18.0.0-3104.g6c07bf89.el8.x86_64.rpm 530 kB/s | 91 kB 00:00 2023-03-29T14:01:19.690 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:01:19.691 INFO:teuthology.orchestra.run.smithi017.stdout:Total 526 kB/s | 91 kB 00:00 2023-03-29T14:01:19.691 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:01:19.702 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:01:19.702 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:01:19.702 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-03-29T14:01:19.758 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:01:19.761 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:01:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:01:19.916 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:20.121 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:20.218 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:20.701 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:20.702 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:20.702 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-03-29T14:01:20.702 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:01:20.702 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:20.702 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-03-29T14:01:20.876 DEBUG:teuthology.parallel:result is None 2023-03-29T14:01:20.977 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:20.977 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:20.977 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:01:20.977 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:01:20.977 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:20.977 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:21.152 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install rbd-mirror 2023-03-29T14:01:21.926 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:04:00 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:23.238 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:23.238 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:23.238 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2023-03-29T14:01:23.238 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:23.238 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 3.3 M 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 3.3 M 2023-03-29T14:01:23.239 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 18 M 2023-03-29T14:01:23.240 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:01:23.752 INFO:teuthology.orchestra.run.smithi017.stdout:rbd-mirror-18.0.0-3104.g6c07bf89.el8.x86_64.rpm 6.5 MB/s | 3.3 MB 00:00 2023-03-29T14:01:23.752 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:01:23.753 INFO:teuthology.orchestra.run.smithi017.stdout:Total 6.5 MB/s | 3.3 MB 00:00 2023-03-29T14:01:23.753 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:01:23.767 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:01:23.767 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:01:23.836 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:01:23.839 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:01:24.310 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:01:24.498 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:25.474 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:25.475 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:25.475 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:01:25.475 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:01:25.475 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:25.475 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:25.655 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install rbd-nbd 2023-03-29T14:01:26.421 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:04:05 ago on Wed 29 Mar 2023 01:57:21 PM UTC. 2023-03-29T14:01:27.719 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd x86_64 2:18.0.0-3104.g6c07bf89.el8 ceph 177 k 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2023-03-29T14:01:27.720 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:27.721 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 177 k 2023-03-29T14:01:27.721 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 626 k 2023-03-29T14:01:27.721 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-03-29T14:01:27.898 INFO:teuthology.orchestra.run.smithi017.stdout:rbd-nbd-18.0.0-3104.g6c07bf89.el8.x86_64.rpm 1.0 MB/s | 177 kB 00:00 2023-03-29T14:01:27.899 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-03-29T14:01:27.899 INFO:teuthology.orchestra.run.smithi017.stdout:Total 992 kB/s | 177 kB 00:00 2023-03-29T14:01:27.899 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-03-29T14:01:27.911 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-03-29T14:01:27.912 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-03-29T14:01:27.967 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-03-29T14:01:27.971 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-03-29T14:01:28.093 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-03-29T14:01:28.288 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:28.599 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:29.153 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 1/1 2023-03-29T14:01:29.153 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:29.153 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-03-29T14:01:29.153 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd-2:18.0.0-3104.g6c07bf89.el8.x86_64 2023-03-29T14:01:29.153 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:29.153 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-03-29T14:01:29.346 DEBUG:teuthology.parallel:result is None 2023-03-29T14:01:29.346 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=6c07bf89555b2fa3413019059913f7a1fb12fde5 2023-03-29T14:01:29.489 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-29T14:01:29.512 INFO:teuthology.orchestra.run.smithi017.stdout:18.0.0-3104.g6c07bf89.el8 2023-03-29T14:01:29.513 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3104.g6c07bf89.el8 2023-03-29T14:01:29.513 INFO:teuthology.task.install:The correct ceph version 18.0.0-3104.g6c07bf89 is installed. 2023-03-29T14:01:29.513 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=6c07bf89555b2fa3413019059913f7a1fb12fde5 2023-03-29T14:01:29.650 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-29T14:01:29.675 INFO:teuthology.orchestra.run.smithi089.stdout:18.0.0-3104.g6c07bf89.el8 2023-03-29T14:01:29.675 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3104.g6c07bf89.el8 2023-03-29T14:01:29.675 INFO:teuthology.task.install:The correct ceph version 18.0.0-3104.g6c07bf89 is installed. 2023-03-29T14:01:29.675 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-03-29T14:01:29.676 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:29.676 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-29T14:01:29.706 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:29.706 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-29T14:01:29.745 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-03-29T14:01:29.746 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:29.746 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2023-03-29T14:01:29.773 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-29T14:01:29.841 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:29.841 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2023-03-29T14:01:29.870 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-29T14:01:29.938 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-03-29T14:01:29.938 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:29.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-29T14:01:29.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-29T14:01:30.034 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:30.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-29T14:01:30.062 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-29T14:01:30.130 INFO:teuthology.run_tasks:Running task ceph... 2023-03-29T14:01:30.209 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-03-29T14:01:30.210 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /var/log/ceph 2023-03-29T14:01:30.212 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2023-03-29T14:01:30.238 INFO:tasks.ceph:Disabling ceph logrotate... 2023-03-29T14:01:30.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-29T14:01:30.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-29T14:01:30.307 INFO:tasks.ceph:Creating extra log directories... 2023-03-29T14:01:30.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-29T14:01:30.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-29T14:01:30.388 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-03-29T14:01:30.389 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-03-29T14:01:30.389 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi/7225415', 'branch': 'wip-matanb-crimson-only-send-map', 'description': 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects}', 'email': None, 'first_in_suite': False, 'job_id': '7225415', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2023-03-29_13:23:53-crimson-rados-wip-matanb-crimson-only-send-map-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-only-send-map'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'wip-matanb-crimson-only-send-map', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-matanb-crimson-only-send-map', 'suite_path': '/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/Matan-B/ceph.git', 'suite_sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5', 'targets': {'smithi017.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIadmY4+aNamScujYWIQPsOgfh4+G70+dP8RKTyQRgGoU2n/ITJR08kxjBvxrSYCBG7pkNFBB200De0o8R2O0YZLGd1yJlB1yOjHm3mX4RREJWhz5RTuKZzWfiCAzZpV649ZQVtjrTaK1pwgvn371XkvxLzMKn/2GKF5nigPYTK/W6W7c9TlgVdUCi46l2DodSyr+vEQJSyBYaPYZSnYCdtzDukVw3XK2LLIrTwnIDRfU485RfJG43mcgVFaXUmoOVzJ4VyvMmU55zwP6Wp+fo0adS3pZHGUSwotdAIuVnw0LlTJuxG87kWnaMFHcYfICRF1UbVyG+GJbRGLZaoWNKoOC90KKrhzyzLPC+1amkOvvRFWUxzKhl7ywQVYp/kZrE1u6jD3uV0DyMBaK4kEz72ha1BNrPZDelyBtlWFs8ls8FOgphqT8enGuZUgVPC8al3GCdskGKZ7YSVzvPDMMYbDufEKr3lCyWEw/kMfudWmd1HB5DrNqGHcVUjd3bX8s=', 'smithi089.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpnrzCKG18+tqEhffbf8sPsXMFhzfP5DhDqb74O8/7+fI+6sQX5u+kDaj/svsMXGQg7rDzMCG9F8kXmC+Y+tEOV8NOCMfIvmRByjl+sxBrPw9hu4YzorMOTZfdKbxx550yTtwEaxJvoTrv+KReVP/2y8FcTZjfEcBCrsVDijSEudH652aEJIfluyUE6nVfT2b7hVdQRt8k547dqYG0/s+fR36HLno0Ay2wJnrUoWQ2y8mbEcWMiDOyMg2Nuwi/TZ+7ix6O1qxW74Fgu40WeUIIff2PImtsFduyPuC7NR78aypQjOPKhoPNaL2ehB0YIZNqp0C+u5pc2E2sMkYPWv9zeyq/yNmWqtVHyymXhExhNKmG+OcMOglq3kme9O8rID+xoYYfTJ5CzDyFpKy2QG1M5IkN8UCuEfICNf6axe6EC9UO7L6Qh3DmxF0ZPwGEy+GkpQM+7yaN3cQmupkP82M66NbSC2rb17pVJLehfsuXXaVGlbghj3WvbO2Hcj4GZXU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5', 'wait-for-scrub': False, 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'rados': {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 0, 'delete': 50, 'read': 100, 'rollback': 0, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e3729161eecf8e6d09a4069acbb883cdcd5e9f50', 'timestamp': '2023-03-29_13:23:53', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.803'} 2023-03-29T14:01:30.390 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-29T14:01:30.418 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-29T14:01:30.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/run/ceph 2023-03-29T14:01:30.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2023-03-29T14:01:30.503 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:30.503 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2023-03-29T14:01:30.559 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-29T14:01:30.560 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2023-03-29T14:01:30.617 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 382883 Links: 1 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-03-29 14:01:24.688234663 +0000 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-03-29 13:56:26.959953237 +0000 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-03-29 13:56:26.959953237 +0000 2023-03-29T14:01:30.618 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-03-29T14:01:30.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-29T14:01:30.684 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-03-29T14:01:30.684 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-03-29T14:01:30.684 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000102252 s, 5.0 MB/s 2023-03-29T14:01:30.685 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-29T14:01:30.745 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 388533 Links: 1 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-03-29 14:01:24.688234663 +0000 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-03-29 13:56:27.493948401 +0000 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-03-29 13:56:27.493948401 +0000 2023-03-29T14:01:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-03-29T14:01:30.804 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-29T14:01:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-03-29T14:01:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-03-29T14:01:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000102997 s, 5.0 MB/s 2023-03-29T14:01:30.871 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-29T14:01:30.930 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2023-03-29T14:01:30.989 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-29T14:01:30.989 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:30.989 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 386288 Links: 1 2023-03-29T14:01:30.989 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:30.989 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:30.989 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-03-29 14:01:24.688234663 +0000 2023-03-29T14:01:30.990 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-03-29 13:56:28.005943764 +0000 2023-03-29T14:01:30.990 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-03-29 13:56:28.005943764 +0000 2023-03-29T14:01:30.990 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-03-29T14:01:30.990 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-29T14:01:31.055 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-03-29T14:01:31.055 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-03-29T14:01:31.055 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000102912 s, 5.0 MB/s 2023-03-29T14:01:31.056 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-29T14:01:31.114 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2023-03-29T14:01:31.172 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-29T14:01:31.173 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:31.173 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 386344 Links: 1 2023-03-29T14:01:31.173 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:31.173 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:31.173 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-03-29 14:01:24.688234663 +0000 2023-03-29T14:01:31.174 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-03-29 13:56:28.519939109 +0000 2023-03-29T14:01:31.174 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-03-29 13:56:28.519939109 +0000 2023-03-29T14:01:31.174 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-03-29T14:01:31.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-29T14:01:31.239 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-03-29T14:01:31.239 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-03-29T14:01:31.239 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 9.245e-05 s, 5.5 MB/s 2023-03-29T14:01:31.241 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-29T14:01:31.300 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-03-29T14:01:31.300 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:31.300 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2023-03-29T14:01:31.319 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-29T14:01:31.319 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2023-03-29T14:01:31.378 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-29T14:01:31.378 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:31.379 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 384988 Links: 1 2023-03-29T14:01:31.379 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:31.379 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:31.379 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-03-29 14:01:16.457207762 +0000 2023-03-29T14:01:31.379 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-03-29 13:56:17.127291977 +0000 2023-03-29T14:01:31.379 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-03-29 13:56:17.127291977 +0000 2023-03-29T14:01:31.380 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-03-29T14:01:31.380 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-29T14:01:31.446 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-03-29T14:01:31.446 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-03-29T14:01:31.446 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000240364 s, 2.1 MB/s 2023-03-29T14:01:31.447 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-29T14:01:31.507 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2023-03-29T14:01:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-29T14:01:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:31.565 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 388382 Links: 1 2023-03-29T14:01:31.565 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:31.565 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:31.566 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-03-29 14:01:16.457207762 +0000 2023-03-29T14:01:31.566 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-03-29 13:56:17.628280888 +0000 2023-03-29T14:01:31.566 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-03-29 13:56:17.628280888 +0000 2023-03-29T14:01:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-03-29T14:01:31.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-29T14:01:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-03-29T14:01:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-03-29T14:01:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.00024012 s, 2.1 MB/s 2023-03-29T14:01:31.633 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-29T14:01:31.691 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2023-03-29T14:01:31.749 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-29T14:01:31.749 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:31.749 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 391211 Links: 1 2023-03-29T14:01:31.749 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:31.749 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:31.749 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-03-29 14:01:16.457207762 +0000 2023-03-29T14:01:31.749 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-03-29 13:56:18.132269733 +0000 2023-03-29T14:01:31.750 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-03-29 13:56:18.132269733 +0000 2023-03-29T14:01:31.750 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-03-29T14:01:31.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-29T14:01:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-03-29T14:01:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-03-29T14:01:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000195196 s, 2.6 MB/s 2023-03-29T14:01:31.816 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-29T14:01:31.876 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2023-03-29T14:01:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-29T14:01:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T14:01:31.934 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 390819 Links: 1 2023-03-29T14:01:31.934 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T14:01:31.934 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T14:01:31.934 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-03-29 14:01:16.457207762 +0000 2023-03-29T14:01:31.934 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-03-29 13:56:18.653258201 +0000 2023-03-29T14:01:31.935 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-03-29 13:56:18.653258201 +0000 2023-03-29T14:01:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-03-29T14:01:31.935 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-29T14:01:31.999 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-03-29T14:01:32.000 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-03-29T14:01:32.000 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000204834 s, 2.5 MB/s 2023-03-29T14:01:32.000 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-29T14:01:32.060 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-03-29T14:01:32.060 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-03-29T14:01:32.060 INFO:tasks.ceph:Generating config... 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] debug monc = 20 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] bluestore rocksdb cf = False 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-03-29T14:01:32.061 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] debug ms = 1 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] debug osd = 20 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] err to stderr = True 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] log flush on exit = True 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] log to file = False 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] log to stderr = True 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] osd delete sleep = 1 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] osd max backfills = 3 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-03-29T14:01:32.062 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-03-29T14:01:32.063 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-03-29T14:01:32.063 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-03-29T14:01:32.063 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-03-29T14:01:32.063 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] ms client mode = crc 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] ms cluster mode = crc 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] ms mon client mode = crc 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] ms mon service mode = crc 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] ms service mode = crc 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-03-29T14:01:32.063 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-03-29T14:01:32.063 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-03-29T14:01:32.063 INFO:tasks.ceph:[mgr] debug ms = 1 2023-03-29T14:01:32.063 INFO:tasks.ceph:[mon] debug mon = 20 2023-03-29T14:01:32.063 INFO:tasks.ceph:[mon] debug ms = 1 2023-03-29T14:01:32.063 INFO:tasks.ceph:[mon] debug paxos = 20 2023-03-29T14:01:32.064 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-03-29T14:01:32.064 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-03-29T14:01:32.064 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-03-29T14:01:32.064 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-03-29T14:01:32.064 INFO:tasks.ceph:[mon] osd pool default crimson = True 2023-03-29T14:01:32.064 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-03-29T14:01:32.064 INFO:tasks.ceph:Setting up mon.a... 2023-03-29T14:01:32.064 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-03-29T14:01:32.135 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.keyring 2023-03-29T14:01:32.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-03-29T14:01:32.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-29T14:01:32.220 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.17')] 2023-03-29T14:01:32.221 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.17', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2023-03-29T14:01:32.222 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:32.222 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-03-29T14:01:32.278 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.17 --print /home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:32.370+0000 7fb048bd3080 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:32.370+0000 7fb048bd3080 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:32.370 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:32.371+0000 7fb048bd3080 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: generated fsid fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = pacific 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 0 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:fsid fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:last_changed 2023-03-29T14:01:32.371999+0000 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:created 2023-03-29T14:01:32.371999+0000 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:min_mon_release 16 (pacific) 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:election_strategy: 1 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2023-03-29T14:01:32.371 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-03-29T14:01:32.372 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-03-29T14:01:32.390 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a... 2023-03-29T14:01:32.391 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-29T14:01:32.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-29T14:01:32.485 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2023-03-29T14:01:32.486 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2023-03-29T14:01:32.487 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon_allow_pool_size_one = true 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = 172.21.15.17 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: ms client mode = crc 2023-03-29T14:01:32.488 INFO:teuthology.orchestra.run.smithi017.stdout: ms cluster mode = crc 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: ms mon client mode = crc 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: ms mon cluster mode = crc 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: ms mon service mode = crc 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: ms service mode = crc 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_min_size = 2 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_size = 3 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 120 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2023-03-29T14:01:32.489 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2023-03-29T14:01:32.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.491 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2023-03-29T14:01:32.491 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2023-03-29T14:01:32.491 INFO:teuthology.orchestra.run.smithi017.stdout: debug monc = 20 2023-03-29T14:01:32.491 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2023-03-29T14:01:32.491 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression algorithm = snappy 2023-03-29T14:01:32.491 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2023-03-29T14:01:32.492 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2023-03-29T14:01:32.492 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore rocksdb cf = False 2023-03-29T14:01:32.492 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 20 2023-03-29T14:01:32.492 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 20 2023-03-29T14:01:32.492 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2023-03-29T14:01:32.492 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2023-03-29T14:01:32.492 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 10 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: err to stderr = True 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: log flush on exit = True 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: log to file = False 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: log to stderr = True 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: osd blocked scrub grace period = 3600 2023-03-29T14:01:32.493 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug reject backfill probability = 0.3 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd delete sleep = 1 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd max backfills = 3 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd max markdown count = 1000 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub min interval = 60 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd shutdown pgref assert = True 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: osd snap trim sleep = 2 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2023-03-29T14:01:32.494 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2023-03-29T14:01:32.495 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: # 1m isn't quite enough 2023-03-29T14:01:32.496 INFO:teuthology.orchestra.run.smithi017.stdout: mon_down_mkfs_grace = 2m 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_filestore_osds = false 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: mon min osdmap epochs = 50 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: mon osdmap full prune interval = 2 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: mon osdmap full prune min = 15 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: mon osdmap full prune txsize = 2 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default crimson = True 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: paxos service trim min = 10 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-29T14:01:32.497 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.528 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2023-03-29T14:01:32.529 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2023-03-29T14:01:32.530 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = 172.21.15.17 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: ms client mode = crc 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: ms cluster mode = crc 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: ms mon client mode = crc 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: ms mon cluster mode = crc 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: ms mon service mode = crc 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: ms service mode = crc 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_min_size = 2 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_size = 3 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2023-03-29T14:01:32.531 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 120 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.532 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: debug monc = 20 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore compression algorithm = snappy 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore compression mode = aggressive 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore rocksdb cf = False 2023-03-29T14:01:32.533 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 20 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 20 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 10 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: err to stderr = True 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: log flush on exit = True 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: log to file = False 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: log to stderr = True 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: osd blocked scrub grace period = 3600 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug reject backfill probability = 0.3 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: osd delete sleep = 1 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: osd max backfills = 3 2023-03-29T14:01:32.534 INFO:teuthology.orchestra.run.smithi089.stdout: osd max markdown count = 1000 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub min interval = 60 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: osd shutdown pgref assert = True 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: osd snap trim sleep = 2 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2023-03-29T14:01:32.535 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.536 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon min osdmap epochs = 50 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon osdmap full prune interval = 2 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon osdmap full prune min = 15 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: mon osdmap full prune txsize = 2 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default crimson = True 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: paxos service trim min = 10 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2023-03-29T14:01:32.537 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2023-03-29T14:01:32.538 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-29T14:01:32.538 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-29T14:01:32.538 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2023-03-29T14:01:32.539 INFO:tasks.ceph:Creating admin key on mon.a... 2023-03-29T14:01:32.539 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-03-29T14:01:32.606 INFO:tasks.ceph:Copying monmap to all nodes... 2023-03-29T14:01:32.606 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:32.607 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-03-29T14:01:32.624 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:32.625 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-03-29T14:01:32.682 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi017.front.sepia.ceph.com 2023-03-29T14:01:32.683 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:32.683 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-29T14:01:32.683 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-29T14:01:32.761 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:32.761 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:32.819 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2023-03-29T14:01:32.819 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:32.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-29T14:01:32.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-29T14:01:32.856 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:32.856 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:32.914 INFO:tasks.ceph:Setting up mon nodes... 2023-03-29T14:01:32.914 INFO:tasks.ceph:Setting up mgr nodes... 2023-03-29T14:01:32.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-03-29T14:01:33.022 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-03-29T14:01:33.025 INFO:tasks.ceph:Setting up mds nodes... 2023-03-29T14:01:33.025 INFO:tasks.ceph_client:Setting up client nodes... 2023-03-29T14:01:33.025 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-03-29T14:01:33.077 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-03-29T14:01:33.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-03-29T14:01:33.141 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-03-29T14:01:33.154 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-03-29T14:01:33.154 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-03-29T14:01:33.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-03-29T14:01:33.182 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-03-29T14:01:33.182 INFO:tasks.ceph:role: osd.0 2023-03-29T14:01:33.182 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com 2023-03-29T14:01:33.182 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T14:01:33.250 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T14:01:33.272 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2023-03-29T14:01:33.274 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2023-03-29T14:01:33.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-03-29T14:01:33.356 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-03-29T14:01:33.430 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-03-29T14:01:33.498 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-03-29T14:01:33.498 INFO:tasks.ceph:role: osd.1 2023-03-29T14:01:33.498 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com 2023-03-29T14:01:33.498 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T14:01:33.567 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T14:01:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T14:01:33.568 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T14:01:33.589 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2023-03-29T14:01:33.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2023-03-29T14:01:33.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-03-29T14:01:33.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-03-29T14:01:33.743 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:34.619 INFO:teuthology.orchestra.run.smithi017.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:34.620 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:34,621 seastar - Reactor backend: epoll 2023-03-29T14:01:34.628 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:34,629 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:34.628 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:34,629 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:34.628 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:34,629 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:34.628 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:34,629 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:==105138==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:==105138==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffedf714000; bottom 0x7fa734a11000; size: 0x0057aad03000 (376527925248) 2023-03-29T14:01:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:False positive error reports may follow 2023-03-29T14:01:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:34,677 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-03-29T14:01:34.677 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:34,677 [shard 0] osd - get_early_config: set --smp 1 2023-03-29T14:01:35.192 INFO:teuthology.orchestra.run.smithi017.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:35.193 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:35,194 seastar - Reactor backend: epoll 2023-03-29T14:01:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:35,201 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:35,202 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:35,202 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:35.201 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:35,202 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:35.203 INFO:teuthology.orchestra.run.smithi017.stderr:==105136==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:==105136==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffedf714000; bottom 0x7fa734a0a000; size: 0x0057aad0a000 (376527953920) 2023-03-29T14:01:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:False positive error reports may follow 2023-03-29T14:01:35.209 INFO:teuthology.orchestra.run.smithi017.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:39.146 INFO:teuthology.orchestra.run.smithi017.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:39.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-03-29T14:01:39.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:40.103 INFO:teuthology.orchestra.run.smithi017.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,105 seastar - Reactor backend: epoll 2023-03-29T14:01:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,113 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,113 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,113 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:40.112 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,113 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:40.114 INFO:teuthology.orchestra.run.smithi017.stderr:==105331==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:40.121 INFO:teuthology.orchestra.run.smithi017.stderr:==105331==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe1014d000; bottom 0x7f8ccf3c6000; size: 0x007140d87000 (486419230720) 2023-03-29T14:01:40.121 INFO:teuthology.orchestra.run.smithi017.stderr:False positive error reports may follow 2023-03-29T14:01:40.121 INFO:teuthology.orchestra.run.smithi017.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:40.160 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,160 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-03-29T14:01:40.160 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,160 [shard 0] osd - get_early_config: set --smp 1 2023-03-29T14:01:40.679 INFO:teuthology.orchestra.run.smithi017.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,681 seastar - Reactor backend: epoll 2023-03-29T14:01:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,688 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,688 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,688 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:INFO 2023-03-29 14:01:40,689 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:==105329==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:40.696 INFO:teuthology.orchestra.run.smithi017.stderr:==105329==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe1014d000; bottom 0x7f8ccf3bf000; size: 0x007140d8e000 (486419259392) 2023-03-29T14:01:40.696 INFO:teuthology.orchestra.run.smithi017.stderr:False positive error reports may follow 2023-03-29T14:01:40.696 INFO:teuthology.orchestra.run.smithi017.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:44.616 INFO:teuthology.orchestra.run.smithi017.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:44.707 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-03-29T14:01:44.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-03-29T14:01:44.765 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-03-29T14:01:44.765 INFO:tasks.ceph:role: osd.2 2023-03-29T14:01:44.766 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2023-03-29T14:01:44.766 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-29T14:01:44.841 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T14:01:44.841 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T14:01:44.841 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T14:01:44.842 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T14:01:44.842 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T14:01:44.842 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-03-29T14:01:44.842 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T14:01:44.842 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T14:01:44.842 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T14:01:44.842 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T14:01:45.265 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-03-29T14:01:45.267 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-03-29T14:01:45.267 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2023-03-29T14:01:45.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-03-29T14:01:45.467 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-03-29T14:01:45.535 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-03-29T14:01:45.535 INFO:tasks.ceph:role: osd.3 2023-03-29T14:01:45.535 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com 2023-03-29T14:01:45.536 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T14:01:45.610 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T14:01:46.040 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-03-29T14:01:46.041 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-03-29T14:01:46.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2023-03-29T14:01:46.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-03-29T14:01:46.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:46,984 seastar - Reactor backend: epoll 2023-03-29T14:01:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:46,995 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:46,995 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:46,995 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:46,995 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:46.997 INFO:teuthology.orchestra.run.smithi089.stderr:==104960==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:47.003 INFO:teuthology.orchestra.run.smithi089.stderr:==104960==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe7a775000; bottom 0x7f003f834000; size: 0x00fe3af41000 (1091910766592) 2023-03-29T14:01:47.003 INFO:teuthology.orchestra.run.smithi089.stderr:False positive error reports may follow 2023-03-29T14:01:47.003 INFO:teuthology.orchestra.run.smithi089.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:47,044 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-03-29T14:01:47.044 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:47,045 [shard 0] osd - get_early_config: set --smp 1 2023-03-29T14:01:47.537 INFO:teuthology.orchestra.run.smithi089.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:47.537 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:47,538 seastar - Reactor backend: epoll 2023-03-29T14:01:47.538 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:47,539 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:47,539 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:47,540 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:47,540 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:47.540 INFO:teuthology.orchestra.run.smithi089.stderr:==104958==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:==104958==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe7a775000; bottom 0x7f003f82d000; size: 0x00fe3af48000 (1091910795264) 2023-03-29T14:01:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:False positive error reports may follow 2023-03-29T14:01:47.547 INFO:teuthology.orchestra.run.smithi089.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:51.451 INFO:teuthology.orchestra.run.smithi089.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:51.511 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-03-29T14:01:51.539 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,329 seastar - Reactor backend: epoll 2023-03-29T14:01:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,342 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,342 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,342 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,342 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:==105152==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:==105152==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd74fd7000; bottom 0x7fda84dd9000; size: 0x0022f01fe000 (150057508864) 2023-03-29T14:01:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:False positive error reports may follow 2023-03-29T14:01:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:52.386 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,387 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-03-29T14:01:52.386 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,387 [shard 0] osd - get_early_config: set --smp 1 2023-03-29T14:01:52.888 INFO:teuthology.orchestra.run.smithi089.stderr:WARNING: debug mode. Not for benchmarking or production 2023-03-29T14:01:52.888 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,889 seastar - Reactor backend: epoll 2023-03-29T14:01:52.890 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,891 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-03-29T14:01:52.890 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,891 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-03-29T14:01:52.890 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,891 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-03-29T14:01:52.890 INFO:teuthology.orchestra.run.smithi089.stderr:INFO 2023-03-29 14:01:52,891 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-03-29T14:01:52.891 INFO:teuthology.orchestra.run.smithi089.stderr:==105150==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-03-29T14:01:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:==105150==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffd74fd7000; bottom 0x7fda84dd2000; size: 0x0022f0205000 (150057537536) 2023-03-29T14:01:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:False positive error reports may follow 2023-03-29T14:01:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-03-29T14:01:56.796 INFO:teuthology.orchestra.run.smithi089.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:01:56.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-03-29T14:01:56.908 INFO:tasks.ceph:Reading keys from all nodes... 2023-03-29T14:01:56.909 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:56.909 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-03-29T14:01:56.946 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:56.946 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-03-29T14:01:57.020 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:57.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-03-29T14:01:57.047 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:57.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-03-29T14:01:57.113 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:57.113 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-03-29T14:01:57.180 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:57.180 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-03-29T14:01:57.203 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:57.203 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-03-29T14:01:57.238 INFO:tasks.ceph:Adding keys to all mons... 2023-03-29T14:01:57.238 DEBUG:teuthology.orchestra.run.smithi017:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.0] 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQC/RCRk2K/iDxAA6wkLL3z593HCxt2/M2M3+Q== 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.1] 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDERCRkLiDILBAAW7rYrqKE6MpKULByaYYSaw== 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.x] 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQC9RCRkZIVeARAAcZYjvWCELlr88yAUb65thw== 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.2] 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDLRCRkd0u/IxAAGwEmhJcZe5UTgdh+fqqQ0w== 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.3] 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDQRCRkzFOhOBAAq7rq+d7jebUTp0ss9JS8Bg== 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2023-03-29T14:01:57.276 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQC9RCRkNb2qBBAAkwpFLO88OoP+kGAVx9iJmA== 2023-03-29T14:01:57.277 INFO:teuthology.orchestra.run.smithi017.stdout:[client.1] 2023-03-29T14:01:57.277 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQC9RCRkmmp6CBAAchlm9RmXAAa0+ofKhSE9Vw== 2023-03-29T14:01:57.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T14:01:57.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T14:01:57.453 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-29T14:01:57.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T14:01:57.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T14:01:57.648 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T14:01:57.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T14:01:57.778 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-03-29T14:01:57.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-03-29T14:01:57.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-29T14:01:57.915 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:57.914+0000 7f213de64c40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:57.915 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:57.914+0000 7f213de64c40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:57.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-03-29T14:01:57.958 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-29T14:01:58.016 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-03-29T14:01:58.016 INFO:tasks.ceph.mon.a:Restarting daemon 2023-03-29T14:01:58.017 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-03-29T14:01:58.059 INFO:tasks.ceph.mon.a:Started 2023-03-29T14:01:58.059 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-03-29T14:01:58.059 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-03-29T14:01:58.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-03-29T14:01:58.061 INFO:tasks.ceph.mgr.x:Started 2023-03-29T14:01:58.062 INFO:tasks.ceph:Setting crush tunables to default 2023-03-29T14:01:58.062 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd crush tunables default 2023-03-29T14:01:58.144 INFO:tasks.ceph.mon.a.smithi017.stderr:2023-03-29T14:01:58.143+0000 7f892bc51c40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.144 INFO:tasks.ceph.mon.a.smithi017.stderr:2023-03-29T14:01:58.143+0000 7f892bc51c40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.145 INFO:tasks.ceph.mon.a.smithi017.stderr:2023-03-29T14:01:58.144+0000 7f892bc51c40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.145 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:01:58.145+0000 7fcaea3e51c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:58.189+0000 7f103f359700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:58.206+0000 7f103f359700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2023-03-29T14:01:58.303 INFO:tasks.ceph:check_enable_crimson: True 2023-03-29T14:01:58.303 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2023-03-29T14:01:58.303 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2023-03-29T14:01:58.348 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:01:58.348+0000 7fcaea3e51c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.349 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:01:58.348+0000 7fcaea3e51c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:58.427+0000 7f639596c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:01:58.435+0000 7f639596c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:58.530 INFO:tasks.ceph.mon.a.smithi017.stderr:2023-03-29T14:01:58.529+0000 7f891ef13700 -1 WARNING: experimental feature 'crimson' is enabled 2023-03-29T14:01:58.530 INFO:tasks.ceph.mon.a.smithi017.stderr:Please be aware that this feature is experimental, untested, 2023-03-29T14:01:58.530 INFO:tasks.ceph.mon.a.smithi017.stderr:unsupported, and may result in data corruption, data loss, 2023-03-29T14:01:58.530 INFO:tasks.ceph.mon.a.smithi017.stderr:and/or irreparable damage to your cluster. Do not use 2023-03-29T14:01:58.530 INFO:tasks.ceph.mon.a.smithi017.stderr:feature with important data. 2023-03-29T14:01:58.530 INFO:tasks.ceph.mon.a.smithi017.stderr: 2023-03-29T14:01:59.226 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:01:59.226+0000 7fcaea3e51c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-29T14:01:59.296 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:01:59.296+0000 7fcaea3e51c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-29T14:01:59.306 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-03-29T14:01:59.306 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:59.306 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-03-29T14:01:59.336 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-03-29T14:01:59.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-03-29T14:01:59.383 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:01:59.382+0000 7fcaea3e51c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-29T14:01:59.405 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:59.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-03-29T14:01:59.433 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-03-29T14:01:59.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-03-29T14:01:59.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 383211ec-0c99-455e-a372-054bfe943fa1 0 2023-03-29T14:01:59.579 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:01:59.580+0000 7fcaea3e51c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-29T14:01:59.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:01:59.661+0000 7f75f87c8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:59.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:01:59.669+0000 7f75f87c8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:59.759 INFO:teuthology.orchestra.run.smithi089.stdout:0 2023-03-29T14:01:59.772 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 44e6d1bf-009e-4ebd-acce-e86c740138ab 1 2023-03-29T14:01:59.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:01:59.936+0000 7f2e6b29f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:01:59.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:01:59.943+0000 7f2e6b29f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:00.035 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-03-29T14:02:00.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 75d54e57-1de1-4211-bdfd-1258692cb78a 2 2023-03-29T14:02:00.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:02:00.171+0000 7f3208c0a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:00.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:02:00.179+0000 7f3208c0a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:00.323 INFO:teuthology.orchestra.run.smithi089.stdout:2 2023-03-29T14:02:00.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 0ec51558-20b8-46b0-9adf-cea66599c494 3 2023-03-29T14:02:00.408 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:00.407+0000 7fcaea3e51c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-29T14:02:00.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:02:00.458+0000 7f7dff258700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-03-29T14:02:00.465+0000 7f7dff258700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:00.557 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:00.557+0000 7fcaea3e51c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-29T14:02:00.558 INFO:teuthology.orchestra.run.smithi089.stdout:3 2023-03-29T14:02:00.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2023-03-29T14:02:00.611 INFO:tasks.ceph.osd.0:Restarting daemon 2023-03-29T14:02:00.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2023-03-29T14:02:00.654 INFO:tasks.ceph.osd.0:Started 2023-03-29T14:02:00.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2023-03-29T14:02:00.691 INFO:tasks.ceph.osd.1:Restarting daemon 2023-03-29T14:02:00.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2023-03-29T14:02:00.734 INFO:tasks.ceph.osd.1:Started 2023-03-29T14:02:00.735 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2023-03-29T14:02:00.768 INFO:tasks.ceph.osd.2:Restarting daemon 2023-03-29T14:02:00.769 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2023-03-29T14:02:00.785 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:00.785+0000 7fcaea3e51c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-29T14:02:00.786 INFO:tasks.ceph.osd.2:Started 2023-03-29T14:02:00.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2023-03-29T14:02:00.816 INFO:tasks.ceph.osd.3:Restarting daemon 2023-03-29T14:02:00.817 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2023-03-29T14:02:00.858 INFO:tasks.ceph.osd.3:Started 2023-03-29T14:02:00.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T14:02:00.868 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:00.869+0000 7fcaea3e51c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-29T14:02:01.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:01.007+0000 7fac90b2c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:01.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:01.016+0000 7fac90b2c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:01.029 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:01.029+0000 7fcaea3e51c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-29T14:02:01.098 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:01.098 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"fsid":"fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a","created":"2023-03-29T14:01:58.194896+0000","modified":"2023-03-29T14:02:00.557643+0000","last_up_change":"0.000000","last_in_change":"2023-03-29T14:02:00.557643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"383211ec-0c99-455e-a372-054bfe943fa1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"44e6d1bf-009e-4ebd-acce-e86c740138ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"75d54e57-1de1-4211-bdfd-1258692cb78a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0ec51558-20b8-46b0-9adf-cea66599c494","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T14:02:01.116 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-03-29T14:02:01.117 INFO:tasks.ceph:Waiting for OSDs to come up 2023-03-29T14:02:01.462 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:01.463+0000 7fcaea3e51c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-29T14:02:01.520 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-29T14:02:01.548 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:01.548+0000 7fcaea3e51c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-29T14:02:01.646 INFO:teuthology.misc.health.smithi017.stderr:2023-03-29T14:02:01.642+0000 7f7b395b2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:01.649 INFO:teuthology.misc.health.smithi017.stderr:2023-03-29T14:02:01.648+0000 7f7b395b2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:01.704 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:01.705+0000 7fcaea3e51c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-29T14:02:01.782 INFO:teuthology.misc.health.smithi017.stdout: 2023-03-29T14:02:01.782 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":7,"fsid":"fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a","created":"2023-03-29T14:01:58.194896+0000","modified":"2023-03-29T14:02:00.557643+0000","last_up_change":"0.000000","last_in_change":"2023-03-29T14:02:00.557643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"383211ec-0c99-455e-a372-054bfe943fa1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"44e6d1bf-009e-4ebd-acce-e86c740138ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"75d54e57-1de1-4211-bdfd-1258692cb78a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0ec51558-20b8-46b0-9adf-cea66599c494","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T14:02:01.794 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-03-29T14:02:01.824 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:01.825+0000 7fcaea3e51c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-29T14:02:01.980 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:01.981+0000 7fcaea3e51c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-29T14:02:02.215 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:02.216+0000 7fcaea3e51c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-29T14:02:02.304 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:02.305+0000 7fcaea3e51c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-29T14:02:02.391 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:02.391+0000 7fcaea3e51c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-29T14:02:03.225 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:03.225+0000 7fcaea3e51c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-29T14:02:03.310 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:03.311+0000 7fcaea3e51c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-29T14:02:03.394 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:03.395+0000 7fcaea3e51c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-29T14:02:03.486 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:03.487+0000 7fcaea3e51c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-29T14:02:03.650 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:03.651+0000 7fcaea3e51c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-03-29T14:02:03.845 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:03.845+0000 7fcaea3e51c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-29T14:02:03.918 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:03.918+0000 7fcaea3e51c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-29T14:02:04.076 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:04.076+0000 7fcaea3e51c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-29T14:02:05.097 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:02:05.097+0000 7fcaa69c5700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-03-29T14:02:08.197 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-29T14:02:08.327 INFO:teuthology.misc.health.smithi017.stderr:2023-03-29T14:02:08.323+0000 7fcf8eab4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:08.330 INFO:teuthology.misc.health.smithi017.stderr:2023-03-29T14:02:08.329+0000 7fcf8eab4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:08.538 INFO:teuthology.misc.health.smithi017.stdout: 2023-03-29T14:02:08.538 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":10,"fsid":"fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a","created":"2023-03-29T14:01:58.194896+0000","modified":"2023-03-29T14:02:08.175335+0000","last_up_change":"2023-03-29T14:02:06.169424+0000","last_in_change":"2023-03-29T14:02:00.557643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T14:02:08.153526+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"383211ec-0c99-455e-a372-054bfe943fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":106185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":106185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":106185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":106185}]},"public_addr":"172.21.15.17:6800/106185","cluster_addr":"172.21.15.17:6801/106185","heartbeat_back_addr":"172.21.15.17:6804/106185","heartbeat_front_addr":"172.21.15.17:6805/106185","state":["exists","up"]},{"osd":1,"uuid":"44e6d1bf-009e-4ebd-acce-e86c740138ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":106212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":106212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":106212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":106212}]},"public_addr":"172.21.15.17:6802/106212","cluster_addr":"172.21.15.17:6803/106212","heartbeat_back_addr":"172.21.15.17:6807/106212","heartbeat_front_addr":"172.21.15.17:6806/106212","state":["exists","up"]},{"osd":2,"uuid":"75d54e57-1de1-4211-bdfd-1258692cb78a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":105801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6803","nonce":105801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":105801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6807","nonce":105801}]},"public_addr":"172.21.15.89:6802/105801","cluster_addr":"172.21.15.89:6803/105801","heartbeat_back_addr":"172.21.15.89:6806/105801","heartbeat_front_addr":"172.21.15.89:6807/105801","state":["exists","up"]},{"osd":3,"uuid":"0ec51558-20b8-46b0-9adf-cea66599c494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6805","nonce":105828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":105828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":105828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6809","nonce":105828}]},"public_addr":"172.21.15.89:6805/105828","cluster_addr":"172.21.15.89:6804/105828","heartbeat_back_addr":"172.21.15.89:6808/105828","heartbeat_front_addr":"172.21.15.89:6809/105828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T14:02:08.550 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-03-29T14:02:08.551 INFO:tasks.ceph:Creating RBD pool 2023-03-29T14:02:08.551 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd pool create rbd 8 2023-03-29T14:02:08.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:08.675+0000 7f887168b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:08.685+0000 7f887168b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:09.185 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'rbd' created 2023-03-29T14:02:09.198 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-03-29T14:02:09.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:09.327+0000 7efde626a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:09.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:09.340+0000 7efde626a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:10.188 INFO:teuthology.orchestra.run.smithi017.stderr:enabled application 'rbd' on pool 'rbd' 2023-03-29T14:02:10.200 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-03-29T14:02:10.201 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-03-29T14:02:10.201 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-03-29T14:02:10.201 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-03-29T14:02:10.201 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-03-29T14:02:10.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:10.337+0000 7f1b8f2a8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:10.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:10.345+0000 7f1b8f2a8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:10.625 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:10.638 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":4,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":105438},{"type":"v1","addr":"172.21.15.89:6801","nonce":105438}]},"active_addr":"172.21.15.89:6801/105438","active_change":"2023-03-29T14:02:04.090244+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":660097862}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3639992224}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3045183619}]}]} 2023-03-29T14:02:10.643 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-03-29T14:02:10.643 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-03-29T14:02:10.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T14:02:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:10.772+0000 7f201e2e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:10.779+0000 7f201e2e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.053 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:11.054 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":12,"fsid":"fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a","created":"2023-03-29T14:01:58.194896+0000","modified":"2023-03-29T14:02:10.182397+0000","last_up_change":"2023-03-29T14:02:06.169424+0000","last_in_change":"2023-03-29T14:02:00.557643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T14:02:08.153526+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T14:02:08.924626+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"383211ec-0c99-455e-a372-054bfe943fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":106185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":106185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":106185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":106185}]},"public_addr":"172.21.15.17:6800/106185","cluster_addr":"172.21.15.17:6801/106185","heartbeat_back_addr":"172.21.15.17:6804/106185","heartbeat_front_addr":"172.21.15.17:6805/106185","state":["exists","up"]},{"osd":1,"uuid":"44e6d1bf-009e-4ebd-acce-e86c740138ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":106212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":106212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":106212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":106212}]},"public_addr":"172.21.15.17:6802/106212","cluster_addr":"172.21.15.17:6803/106212","heartbeat_back_addr":"172.21.15.17:6807/106212","heartbeat_front_addr":"172.21.15.17:6806/106212","state":["exists","up"]},{"osd":2,"uuid":"75d54e57-1de1-4211-bdfd-1258692cb78a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":105801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6803","nonce":105801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":105801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6807","nonce":105801}]},"public_addr":"172.21.15.89:6802/105801","cluster_addr":"172.21.15.89:6803/105801","heartbeat_back_addr":"172.21.15.89:6806/105801","heartbeat_front_addr":"172.21.15.89:6807/105801","state":["exists","up"]},{"osd":3,"uuid":"0ec51558-20b8-46b0-9adf-cea66599c494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6805","nonce":105828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":105828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":105828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6809","nonce":105828}]},"public_addr":"172.21.15.89:6805/105828","cluster_addr":"172.21.15.89:6804/105828","heartbeat_back_addr":"172.21.15.89:6808/105828","heartbeat_front_addr":"172.21.15.89:6809/105828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T14:02:11.070 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-03-29T14:02:11.071 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T14:02:11.204 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.200+0000 7fad163ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.207+0000 7fad163ca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.423 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:11.423 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"fsid":"fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a","created":"2023-03-29T14:01:58.194896+0000","modified":"2023-03-29T14:02:11.186275+0000","last_up_change":"2023-03-29T14:02:06.169424+0000","last_in_change":"2023-03-29T14:02:00.557643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T14:02:08.153526+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T14:02:08.924626+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"383211ec-0c99-455e-a372-054bfe943fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":106185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":106185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":106185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":106185}]},"public_addr":"172.21.15.17:6800/106185","cluster_addr":"172.21.15.17:6801/106185","heartbeat_back_addr":"172.21.15.17:6804/106185","heartbeat_front_addr":"172.21.15.17:6805/106185","state":["exists","up"]},{"osd":1,"uuid":"44e6d1bf-009e-4ebd-acce-e86c740138ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":106212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":106212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":106212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":106212}]},"public_addr":"172.21.15.17:6802/106212","cluster_addr":"172.21.15.17:6803/106212","heartbeat_back_addr":"172.21.15.17:6807/106212","heartbeat_front_addr":"172.21.15.17:6806/106212","state":["exists","up"]},{"osd":2,"uuid":"75d54e57-1de1-4211-bdfd-1258692cb78a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":105801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6803","nonce":105801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":105801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6807","nonce":105801}]},"public_addr":"172.21.15.89:6802/105801","cluster_addr":"172.21.15.89:6803/105801","heartbeat_back_addr":"172.21.15.89:6806/105801","heartbeat_front_addr":"172.21.15.89:6807/105801","state":["exists","up"]},{"osd":3,"uuid":"0ec51558-20b8-46b0-9adf-cea66599c494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6805","nonce":105828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":105828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":105828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6809","nonce":105828}]},"public_addr":"172.21.15.89:6805/105828","cluster_addr":"172.21.15.89:6804/105828","heartbeat_back_addr":"172.21.15.89:6808/105828","heartbeat_front_addr":"172.21.15.89:6809/105828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T14:02:11.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-03-29T14:02:11.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-03-29T14:02:11.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-03-29T14:02:11.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-03-29T14:02:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.582+0000 7f5a9d6db700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.591 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.585+0000 7f1198758700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.591 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.590+0000 7f5a9d6db700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.586+0000 7f9a8bd0c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.595 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.595+0000 7f1198758700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.595+0000 7f01aed27700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.603 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.603+0000 7f9a8bd0c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.607 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.607+0000 7f01aed27700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.626 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:11.627 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-03-29T14:02:11.633 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:11.633 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-03-29T14:02:11.639 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:11.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-03-29T14:02:11.651 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:11.651 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-03-29T14:02:11.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.778+0000 7f24b04f7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.789 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.788+0000 7f24b04f7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.804 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.797+0000 7f91164f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.810+0000 7f91164f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.846+0000 7ffb4c83c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.846+0000 7f7a2a2ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.855+0000 7f7a2a2ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:11.855+0000 7ffb4c83c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:12.066 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:12.072 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:12.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2023-03-29T14:02:12.082 DEBUG:teuthology.parallel:result is None 2023-03-29T14:02:12.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2023-03-29T14:02:12.089 DEBUG:teuthology.parallel:result is None 2023-03-29T14:02:12.098 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:12.104 INFO:teuthology.orchestra.run.smithi017.stdout:38654705669 2023-03-29T14:02:12.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2023-03-29T14:02:12.117 DEBUG:teuthology.parallel:result is None 2023-03-29T14:02:12.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2023-03-29T14:02:12.122 DEBUG:teuthology.parallel:result is None 2023-03-29T14:02:12.122 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-03-29T14:02:12.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:12.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:12.293+0000 7f185d19e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:12.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:12.303+0000 7f185d19e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:12.518 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:12.533 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-03-29T14:02:12.094656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":20036,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":19648,"kb_avail":377467324,"statfs":{"total":386547056640,"available":386526539776,"internally_reserved":0,"allocated":315392,"data_stored":1056376,"data_compressed":45921,"data_compressed_allocated":61440,"data_compressed_original":958464,"omap_allocated":0,"internal_metadata":20119552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262144,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6,"num_write":28,"num_write_kb":520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.485843"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.274938+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:10.274938+0000","last_peered":"2023-03-29T14:02:10.274938+0000","last_clean":"2023-03-29T14:02:10.274938+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:10.274938+0000","last_undegraded":"2023-03-29T14:02:10.274938+0000","last_fullsized":"2023-03-29T14:02:10.274938+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.353942+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:10.353942+0000","last_peered":"2023-03-29T14:02:10.353942+0000","last_clean":"2023-03-29T14:02:10.353942+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:10.353942+0000","last_undegraded":"2023-03-29T14:02:10.353942+0000","last_fullsized":"2023-03-29T14:02:10.353942+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.374276+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:10.374276+0000","last_peered":"2023-03-29T14:02:10.374276+0000","last_clean":"2023-03-29T14:02:10.374276+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:10.374276+0000","last_undegraded":"2023-03-29T14:02:10.374276+0000","last_fullsized":"2023-03-29T14:02:10.374276+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.367291+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:10.367291+0000","last_peered":"2023-03-29T14:02:10.367291+0000","last_clean":"2023-03-29T14:02:10.367291+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:10.367291+0000","last_undegraded":"2023-03-29T14:02:10.367291+0000","last_fullsized":"2023-03-29T14:02:10.367291+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'50","reported_seq":131,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.573177+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:11.573177+0000","last_peered":"2023-03-29T14:02:11.573177+0000","last_clean":"2023-03-29T14:02:11.573177+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:11.573177+0000","last_undegraded":"2023-03-29T14:02:11.573177+0000","last_fullsized":"2023-03-29T14:02:11.573177+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.316305+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:10.316305+0000","last_peered":"2023-03-29T14:02:10.316305+0000","last_clean":"2023-03-29T14:02:10.316305+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:10.316305+0000","last_undegraded":"2023-03-29T14:02:10.316305+0000","last_fullsized":"2023-03-29T14:02:10.316305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.333140+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:10.333140+0000","last_peered":"2023-03-29T14:02:10.333140+0000","last_clean":"2023-03-29T14:02:10.333140+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:10.333140+0000","last_undegraded":"2023-03-29T14:02:10.333140+0000","last_fullsized":"2023-03-29T14:02:10.333140+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.375236+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:10.375236+0000","last_peered":"2023-03-29T14:02:10.375236+0000","last_clean":"2023-03-29T14:02:10.375236+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:10.375236+0000","last_undegraded":"2023-03-29T14:02:10.375236+0000","last_fullsized":"2023-03-29T14:02:10.375236+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.410007+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:10.410007+0000","last_peered":"2023-03-29T14:02:10.410007+0000","last_clean":"2023-03-29T14:02:10.410007+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:10.410007+0000","last_undegraded":"2023-03-29T14:02:10.410007+0000","last_fullsized":"2023-03-29T14:02:10.410007+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4860,"kb_used_data":52,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366980,"statfs":{"total":96636764160,"available":96631787520,"internally_reserved":0,"allocated":53248,"data_stored":17281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5100,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5036,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5040,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":90112,"data_stored":348117,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:12.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:12.663+0000 7f2c2e9d8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:12.669+0000 7f2c2e9d8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:12.875 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:12.887 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-03-29T14:02:12.094656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":20036,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":19648,"kb_avail":377467324,"statfs":{"total":386547056640,"available":386526539776,"internally_reserved":0,"allocated":315392,"data_stored":1056376,"data_compressed":45921,"data_compressed_allocated":61440,"data_compressed_original":958464,"omap_allocated":0,"internal_metadata":20119552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262144,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6,"num_write":28,"num_write_kb":520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.485843"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.274938+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:10.274938+0000","last_peered":"2023-03-29T14:02:10.274938+0000","last_clean":"2023-03-29T14:02:10.274938+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:10.274938+0000","last_undegraded":"2023-03-29T14:02:10.274938+0000","last_fullsized":"2023-03-29T14:02:10.274938+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.353942+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:10.353942+0000","last_peered":"2023-03-29T14:02:10.353942+0000","last_clean":"2023-03-29T14:02:10.353942+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:10.353942+0000","last_undegraded":"2023-03-29T14:02:10.353942+0000","last_fullsized":"2023-03-29T14:02:10.353942+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.374276+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:10.374276+0000","last_peered":"2023-03-29T14:02:10.374276+0000","last_clean":"2023-03-29T14:02:10.374276+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:10.374276+0000","last_undegraded":"2023-03-29T14:02:10.374276+0000","last_fullsized":"2023-03-29T14:02:10.374276+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.367291+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:10.367291+0000","last_peered":"2023-03-29T14:02:10.367291+0000","last_clean":"2023-03-29T14:02:10.367291+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:10.367291+0000","last_undegraded":"2023-03-29T14:02:10.367291+0000","last_fullsized":"2023-03-29T14:02:10.367291+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'50","reported_seq":131,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.573177+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:11.573177+0000","last_peered":"2023-03-29T14:02:11.573177+0000","last_clean":"2023-03-29T14:02:11.573177+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:11.573177+0000","last_undegraded":"2023-03-29T14:02:11.573177+0000","last_fullsized":"2023-03-29T14:02:11.573177+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.316305+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:10.316305+0000","last_peered":"2023-03-29T14:02:10.316305+0000","last_clean":"2023-03-29T14:02:10.316305+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:10.316305+0000","last_undegraded":"2023-03-29T14:02:10.316305+0000","last_fullsized":"2023-03-29T14:02:10.316305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.333140+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:10.333140+0000","last_peered":"2023-03-29T14:02:10.333140+0000","last_clean":"2023-03-29T14:02:10.333140+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:10.333140+0000","last_undegraded":"2023-03-29T14:02:10.333140+0000","last_fullsized":"2023-03-29T14:02:10.333140+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.375236+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:10.375236+0000","last_peered":"2023-03-29T14:02:10.375236+0000","last_clean":"2023-03-29T14:02:10.375236+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:10.375236+0000","last_undegraded":"2023-03-29T14:02:10.375236+0000","last_fullsized":"2023-03-29T14:02:10.375236+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.410007+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:10.410007+0000","last_peered":"2023-03-29T14:02:10.410007+0000","last_clean":"2023-03-29T14:02:10.410007+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:10.410007+0000","last_undegraded":"2023-03-29T14:02:10.410007+0000","last_fullsized":"2023-03-29T14:02:10.410007+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4860,"kb_used_data":52,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366980,"statfs":{"total":96636764160,"available":96631787520,"internally_reserved":0,"allocated":53248,"data_stored":17281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5100,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5036,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5040,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":90112,"data_stored":348117,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:12.888 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-03-29T14:02:12.888 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-03-29T14:02:12.889 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-03-29T14:02:12.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-03-29T14:02:13.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:13.029+0000 7f383f6c0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:13.035 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:13.034+0000 7f383f6c0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:13.288 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-03-29T14:02:13.302 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-03-29T14:02:13.302 INFO:teuthology.run_tasks:Running task thrashosds... 2023-03-29T14:02:13.312 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2023-03-29T14:02:13.312 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-matanb-crimson-only-send-map'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'wip-matanb-crimson-only-send-map', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '6c07bf89555b2fa3413019059913f7a1fb12fde5'}} 2023-03-29T14:02:13.312 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2023-03-29T14:02:13.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-03-29T14:02:13.447 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:13.439+0000 7f1f64c31700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:13.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:13.450+0000 7f1f64c31700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:13.672 INFO:tasks.thrashosds:Beginning thrashosds... 2023-03-29T14:02:13.672 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-03-29T14:02:13.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:13.806 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:13.802+0000 7f07a5392700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:13.809 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:13.808+0000 7f07a5392700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:14.010 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:14.013 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:14.025 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-03-29T14:02:12.094656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":20036,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":19648,"kb_avail":377467324,"statfs":{"total":386547056640,"available":386526539776,"internally_reserved":0,"allocated":315392,"data_stored":1056376,"data_compressed":45921,"data_compressed_allocated":61440,"data_compressed_original":958464,"omap_allocated":0,"internal_metadata":20119552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262144,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6,"num_write":28,"num_write_kb":520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.485843"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.274938+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:10.274938+0000","last_peered":"2023-03-29T14:02:10.274938+0000","last_clean":"2023-03-29T14:02:10.274938+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:10.274938+0000","last_undegraded":"2023-03-29T14:02:10.274938+0000","last_fullsized":"2023-03-29T14:02:10.274938+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.353942+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:10.353942+0000","last_peered":"2023-03-29T14:02:10.353942+0000","last_clean":"2023-03-29T14:02:10.353942+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:10.353942+0000","last_undegraded":"2023-03-29T14:02:10.353942+0000","last_fullsized":"2023-03-29T14:02:10.353942+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.374276+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:10.374276+0000","last_peered":"2023-03-29T14:02:10.374276+0000","last_clean":"2023-03-29T14:02:10.374276+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:10.374276+0000","last_undegraded":"2023-03-29T14:02:10.374276+0000","last_fullsized":"2023-03-29T14:02:10.374276+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.367291+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:10.367291+0000","last_peered":"2023-03-29T14:02:10.367291+0000","last_clean":"2023-03-29T14:02:10.367291+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:10.367291+0000","last_undegraded":"2023-03-29T14:02:10.367291+0000","last_fullsized":"2023-03-29T14:02:10.367291+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'50","reported_seq":131,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.573177+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:11.573177+0000","last_peered":"2023-03-29T14:02:11.573177+0000","last_clean":"2023-03-29T14:02:11.573177+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:11.573177+0000","last_undegraded":"2023-03-29T14:02:11.573177+0000","last_fullsized":"2023-03-29T14:02:11.573177+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.316305+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:10.316305+0000","last_peered":"2023-03-29T14:02:10.316305+0000","last_clean":"2023-03-29T14:02:10.316305+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:10.316305+0000","last_undegraded":"2023-03-29T14:02:10.316305+0000","last_fullsized":"2023-03-29T14:02:10.316305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.333140+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:10.333140+0000","last_peered":"2023-03-29T14:02:10.333140+0000","last_clean":"2023-03-29T14:02:10.333140+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:10.333140+0000","last_undegraded":"2023-03-29T14:02:10.333140+0000","last_fullsized":"2023-03-29T14:02:10.333140+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.375236+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:10.375236+0000","last_peered":"2023-03-29T14:02:10.375236+0000","last_clean":"2023-03-29T14:02:10.375236+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:10.375236+0000","last_undegraded":"2023-03-29T14:02:10.375236+0000","last_fullsized":"2023-03-29T14:02:10.375236+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.410007+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:10.410007+0000","last_peered":"2023-03-29T14:02:10.410007+0000","last_clean":"2023-03-29T14:02:10.410007+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:10.410007+0000","last_undegraded":"2023-03-29T14:02:10.410007+0000","last_fullsized":"2023-03-29T14:02:10.410007+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4860,"kb_used_data":52,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366980,"statfs":{"total":96636764160,"available":96631787520,"internally_reserved":0,"allocated":53248,"data_stored":17281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5100,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5036,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5040,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":90112,"data_stored":348117,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:14.027 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:14.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:14.156+0000 7f9592f1c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:14.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:14.162+0000 7f9592f1c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:14.364 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:14.378 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-03-29T14:02:14.095189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":20112,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":19712,"kb_avail":377467248,"statfs":{"total":386547056640,"available":386526461952,"internally_reserved":0,"allocated":327680,"data_stored":1190076,"data_compressed":52313,"data_compressed_allocated":69632,"data_compressed_original":1089536,"omap_allocated":0,"internal_metadata":20185088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262144,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6,"num_write":28,"num_write_kb":520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.486375"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.274938+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:10.274938+0000","last_peered":"2023-03-29T14:02:10.274938+0000","last_clean":"2023-03-29T14:02:10.274938+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:10.274938+0000","last_undegraded":"2023-03-29T14:02:10.274938+0000","last_fullsized":"2023-03-29T14:02:10.274938+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.353942+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:10.353942+0000","last_peered":"2023-03-29T14:02:10.353942+0000","last_clean":"2023-03-29T14:02:10.353942+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:10.353942+0000","last_undegraded":"2023-03-29T14:02:10.353942+0000","last_fullsized":"2023-03-29T14:02:10.353942+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.374276+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:10.374276+0000","last_peered":"2023-03-29T14:02:10.374276+0000","last_clean":"2023-03-29T14:02:10.374276+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:10.374276+0000","last_undegraded":"2023-03-29T14:02:10.374276+0000","last_fullsized":"2023-03-29T14:02:10.374276+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.367291+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:10.367291+0000","last_peered":"2023-03-29T14:02:10.367291+0000","last_clean":"2023-03-29T14:02:10.367291+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:10.367291+0000","last_undegraded":"2023-03-29T14:02:10.367291+0000","last_fullsized":"2023-03-29T14:02:10.367291+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'50","reported_seq":131,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.573177+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:11.573177+0000","last_peered":"2023-03-29T14:02:11.573177+0000","last_clean":"2023-03-29T14:02:11.573177+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:11.573177+0000","last_undegraded":"2023-03-29T14:02:11.573177+0000","last_fullsized":"2023-03-29T14:02:11.573177+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.316305+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:10.316305+0000","last_peered":"2023-03-29T14:02:10.316305+0000","last_clean":"2023-03-29T14:02:10.316305+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:10.316305+0000","last_undegraded":"2023-03-29T14:02:10.316305+0000","last_fullsized":"2023-03-29T14:02:10.316305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.333140+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:10.333140+0000","last_peered":"2023-03-29T14:02:10.333140+0000","last_clean":"2023-03-29T14:02:10.333140+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:10.333140+0000","last_undegraded":"2023-03-29T14:02:10.333140+0000","last_fullsized":"2023-03-29T14:02:10.333140+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.375236+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:10.375236+0000","last_peered":"2023-03-29T14:02:10.375236+0000","last_clean":"2023-03-29T14:02:10.375236+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:10.375236+0000","last_undegraded":"2023-03-29T14:02:10.375236+0000","last_fullsized":"2023-03-29T14:02:10.375236+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.410007+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:10.410007+0000","last_peered":"2023-03-29T14:02:10.410007+0000","last_clean":"2023-03-29T14:02:10.410007+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:10.410007+0000","last_undegraded":"2023-03-29T14:02:10.410007+0000","last_fullsized":"2023-03-29T14:02:10.410007+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4860,"kb_used_data":52,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366980,"statfs":{"total":96636764160,"available":96631787520,"internally_reserved":0,"allocated":53248,"data_stored":17281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5176,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366664,"statfs":{"total":96636764160,"available":96631463936,"internally_reserved":0,"allocated":98304,"data_stored":479189,"data_compressed":21699,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5036,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5040,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":90112,"data_stored":348117,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:14.379 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-03-29T14:02:14.379 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-03-29T14:02:14.513 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:14.508+0000 7f4064461700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:14.517 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:14.515+0000 7f4064461700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:14.722 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 13 2023-03-29T14:02:14.722 INFO:teuthology.orchestra.run.smithi017.stdout:fsid fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:created 2023-03-29T14:01:58.194896+0000 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2023-03-29T14:02:11.186275+0000 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:crush_version 3 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:full_ratio 0.95 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:backfillfull_ratio 0.9 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:nearfull_ratio 0.85 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:require_min_compat_client luminous 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:min_compat_client jewel 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:require_osd_release reef 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:stretch_mode_enabled false 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:allow_crimson=true 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 12 flags hashpspool,nopgchange,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,nopgchange,crimson stripe_width 0 application rbd read_balance_score 2.00 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:max_osd 4 2023-03-29T14:02:14.723 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.17:6800/106185 v2:172.21.15.17:6801/106185 exists,up 383211ec-0c99-455e-a372-054bfe943fa1 2023-03-29T14:02:14.724 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.17:6802/106212 v2:172.21.15.17:6803/106212 exists,up 44e6d1bf-009e-4ebd-acce-e86c740138ab 2023-03-29T14:02:14.724 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.89:6802/105801 v2:172.21.15.89:6803/105801 exists,up 75d54e57-1de1-4211-bdfd-1258692cb78a 2023-03-29T14:02:14.724 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.89:6805/105828 v2:172.21.15.89:6804/105828 exists,up 0ec51558-20b8-46b0-9adf-cea66599c494 2023-03-29T14:02:14.737 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.17:6800/106185 v2:172.21.15.17:6801/106185 exists,up 383211ec-0c99-455e-a372-054bfe943fa1', 'osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.17:6802/106212 v2:172.21.15.17:6803/106212 exists,up 44e6d1bf-009e-4ebd-acce-e86c740138ab', 'osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.89:6802/105801 v2:172.21.15.89:6803/105801 exists,up 75d54e57-1de1-4211-bdfd-1258692cb78a', 'osd.3 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.89:6805/105828 v2:172.21.15.89:6804/105828 exists,up 0ec51558-20b8-46b0-9adf-cea66599c494'] 2023-03-29T14:02:14.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-03-29T14:02:14.869 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-03-29T14:02:14.869 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mon.a", 2023-03-29T14:02:14.869 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster": "ceph", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "admin_socket_mode": "", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_client_required": "cephx, none", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_cluster_required": "cephx", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_debug": "true", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-03-29T14:02:14.870 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_service_required": "cephx", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_service_ticket_ttl": "240.000000", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_supported": "cephx", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio": "true", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio_max_queue_depth": "1024", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio_poll_ms": "250", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio_reap_max": "16", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_async_discard": "false", 2023-03-29T14:02:14.871 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_block_size": "4096", 2023-03-29T14:02:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_aio": "false", 2023-03-29T14:02:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-03-29T14:02:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-03-29T14:02:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_inflight_ios": "false", 2023-03-29T14:02:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_enable_discard": "false", 2023-03-29T14:02:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_flock_retry": "3", 2023-03-29T14:02:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_flock_retry_interval": "0.100000", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_inject_crash": "0", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_inject_crash_flush_delay": "2", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_ioring": "false", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_ioring_hipri": "false", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_ioring_sqthread_poll": "false", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_read_buffer_alignment": "4096", 2023-03-29T14:02:14.873 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_type": "", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_alloc_size": "1048576", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_allocator": "hybrid", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_buffered_io": "true", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_check_for_zeros": "false", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_check_volume_selector_often": "false", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-03-29T14:02:14.874 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_compact_log_sync": "false", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_log_compact_min_size": "16777216", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_log_replay_check_allocations": "true", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_max_log_runway": "4194304", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_max_prefetch": "1048576", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_min_flush_size": "524288", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_min_log_runway": "1048576", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_replay_recovery": "false", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-03-29T14:02:14.875 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_shared_alloc_size": "65536", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_sync_write": "false", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_allocation_from_file": "true", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_allocator": "hybrid", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-03-29T14:02:14.876 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-03-29T14:02:14.877 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-03-29T14:02:14.877 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blobid_prealloc": "10240", 2023-03-29T14:02:14.877 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_create": "true", 2023-03-29T14:02:14.877 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_db_create": "false", 2023-03-29T14:02:14.877 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_db_path": "", 2023-03-29T14:02:14.877 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_db_size": "0", 2023-03-29T14:02:14.877 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_path": "", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_preallocate_file": "false", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_size": "107374182400", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_wal_create": "false", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_wal_path": "", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_wal_size": "100663296", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs": "true", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs_env_mirror": "false", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-03-29T14:02:14.878 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_autotune": "true", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_size": "0", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_type": "2q", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_clone_cow": "true", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_algorithm": "snappy", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_max_blob_size": "0", 2023-03-29T14:02:14.879 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_min_blob_size": "0", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_mode": "none", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_csum_type": "crc32c", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_enforce_settings": "default", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_freelist": "false", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_fsck_abort": "false", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_inject_bug21040": "false", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_inject_read_err": "false", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_legacy_omap": "false", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-03-29T14:02:14.880 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_prefill": "0.000000", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_small_allocations": "0", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_default_buffered_read": "true", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_default_buffered_write": "false", 2023-03-29T14:02:14.881 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_deferred_batch_ops": "0", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fail_eio": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mkfs": "true", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mount": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_umount": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-03-29T14:02:14.882 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_ignore_data_csum": "false", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_kvbackend": "rocksdb", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_log_op_age": "5.000000", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_alloc_size": "0", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_blob_size": "0", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_defer_interval": "3.000000", 2023-03-29T14:02:14.883 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_deferred_txc": "32", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_min_alloc_size": "0", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_nid_prealloc": "1024", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_prefer_deferred_size": "0", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_qfsck_on_mount": "true", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_retry_disk_reads": "3", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_cf": "true", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=128,min_write_buffer_number_to_merge=16,compaction_style=kCompactionStyleLevel,write_buffer_size=8388608,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0,ttl=21600", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_options_annex": "", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_coremask": "0x1", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_io_sleep": "5", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_max_io_completion": "0", 2023-03-29T14:02:14.884 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_mem": "512", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_sync_submit_transaction": "false", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_bytes": "67108864", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_cost_per_io": "0", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_tracing": "false", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_volume_selection_reserved": "0", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-03-29T14:02:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_zero_block_detection": "false", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_action_update_interval": "2", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_mount_timeout": "10", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_cluster_require_signatures": "false", 2023-03-29T14:02:14.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_cluster_require_version": "2", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_require_signatures": "false", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_require_version": "2", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_service_require_signatures": "false", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_service_require_version": "2", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_sign_messages": "true", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "chdir": "", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_acl_type": "", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_asio_thread_count": "2", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_cache_mid": "0.750000", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_cache_size": "16384", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_caps_release_delay": "5", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_check_pool_perm": "true", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_collect_and_send_global_metrics": "false", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_debug_force_sync_read": "false", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_debug_getattr_caps": "false", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_debug_inject_tick_delay": "0", 2023-03-29T14:02:14.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_die_on_failed_remount": "false", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_dirsize_rbytes": "true", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_force_lazyio": "false", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_fs": "", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_inject_fixed_oldest_tid": "false", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_inject_release_failure": "false", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_max_inline_size": "4096", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_max_retries_on_remount_failure": "5", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mds_namespace": "", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_metadata": "", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mount_gid": "-1", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mount_timeout": "300", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mount_uid": "-1", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mountpoint": "/", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_notify_timeout": "10", 2023-03-29T14:02:14.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc": "true", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_max_dirty": "104857600", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_max_dirty_age": "5.000000", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_max_objects": "1000", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_size": "209715200", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_target_dirty": "8388608", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_permissions": "true", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_quota": "true", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_quota_df": "true", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_readahead_max_bytes": "0", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_readahead_max_periods": "4", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_readahead_min": "131072", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_reconnect_stale": "false", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_shutdown_timeout": "30", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_snapdir": ".snap", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_tick_interval": "1", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_trace": "", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_try_dentry_invalidate": "false", 2023-03-29T14:02:14.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_use_faked_inos": "false", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "client_use_random_mds": "false", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_graylog": "false", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_graylog_port": "12201", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_monitors": "default=true", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_syslog": "false", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_syslog_level": "info", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "-", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_network": "", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_network_interface": "", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zlib_isal": "false", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zlib_level": "5", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zlib_winsize": "-15", 2023-03-29T14:02:14.890 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zstd_level": "1", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_alien_op_num_threads": "6", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_alien_thread_cpu_cores": "", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_osd_obc_lru_size": "10", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_seastar_smp": "1", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location_hook": "", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location_hook_timeout": "10", 2023-03-29T14:02:14.891 INFO:teuthology.orchestra.run.smithi017.stdout: "daemonize": "false", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "daos_pool": "tank", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "dbstore_db_name_prefix": "dbstore", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_alienstore": "0/5", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_allow_any_pool_priority": "false", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_asok": "1/5", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_asok_assert_abort": "false", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_asserts_on_shutdown": "true", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_auth": "20/20", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_bdev": "1/3", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_bluefs": "1/5", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_bluestore": "1/5", 2023-03-29T14:02:14.892 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_buffer": "0/1", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_ceph_exporter": "1/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_cephfs_mirror": "0/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_cephsqlite": "0/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_client": "0/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_compressor": "1/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_context": "0/1", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_crush": "1/1", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_crypto": "1/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_cyanstore": "0/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_deliberately_leak_memory": "false", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_disable_randomized_ping": "false", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_dpdk": "1/5", 2023-03-29T14:02:14.893 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_eventtrace": "1/5", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_filer": "0/1", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_filestore": "1/3", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_finisher": "1/1", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_fuse": "1/5", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_heartbeat_testing_span": "0", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_heartbeatmap": "1/5", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_immutable_obj_cache": "0/5", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_javaclient": "1/5", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_journal": "1/3", 2023-03-29T14:02:14.894 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_journaler": "0/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_kstore": "1/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_leveldb": "4/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_lockdep": "0/1", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mclock": "1/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds": "1/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_balancer": "1/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_locker": "1/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_log": "1/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_log_expire": "1/5", 2023-03-29T14:02:14.895 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_migrator": "1/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_memstore": "1/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mgr": "2/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mgrc": "1/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mon": "20/20", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_monc": "0/10", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_ms": "1/1", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_none": "0/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_objclass": "0/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_objectcacher": "0/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_objecter": "0/1", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_optracker": "0/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_osd": "1/5", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_paxos": "20/20", 2023-03-29T14:02:14.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_perfcounter": "1/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_prioritycache": "1/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rados": "0/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd": "0/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd_mirror": "0/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd_pwl": "0/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd_replay": "0/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_refs": "0/0", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_reserver": "1/1", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw": "1/5", 2023-03-29T14:02:14.897 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_access": "1/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_datacache": "1/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_dbstore": "1/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_flight": "1/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_sync": "1/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rocksdb": "4/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_backref": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_cache": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_cleaner": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_device": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_epm": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_journal": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_lba": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_odata": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_omap": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_onode": "0/5", 2023-03-29T14:02:14.898 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_t": "0/5", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_tm": "0/5", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_striper": "0/1", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_test": "0/5", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_throttle": "1/1", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_timer": "0/1", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_tp": "0/5", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "device_failure_prediction_mode": "none", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_graylog": "false", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_journald": "false", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_stderr": "true", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_syslog": "false", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event_tracing": "false", 2023-03-29T14:02:14.899 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_addr": "0.0.0.0", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_http_port": "9926", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_prio_limit": "5", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_sort_metrics": "true", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_stats_period": "5", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "fake_statfs_for_testing": "0", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "fatal_signal_handlers": "true", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "filer_max_purge_ops": "10", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "filer_max_truncate_ops": "128", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_apply_finisher_threads": "1", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_blackhole": "false", 2023-03-29T14:02:14.900 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_btrfs_clone_range": "true", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_btrfs_snap": "true", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_caller_concurrency": "10", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_collect_device_partition_information": "true", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_commit_timeout": "600.000000", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_inject_read_err": "false", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_omap_check": "false", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_random_read_err": "0.000000", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_verify_split": "false", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_dump_file": "", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fadvise": "true", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fail_eio": "true", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fd_cache_shards": "16", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fd_cache_size": "128", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fiemap": "false", 2023-03-29T14:02:14.901 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fiemap_threshold": "4096", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_index_retry_probability": "0.000000", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_inject_stall": "0", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_journal_parallel": "false", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_journal_trailing": "false", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_journal_writeahead": "false", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_kill_at": "0", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size": "0", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs": "0", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs_other": "2", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_sync_interval": "5.000000", 2023-03-29T14:02:14.902 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size": "0", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_merge_threshold": "-10", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_min_sync_interval": "0.010000", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_odsync_write": "false", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_omap_backend": "rocksdb", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_omap_backend_path": "", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_omap_header_cache_size": "1024", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_ondisk_finisher_threads": "1", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_op_thread_timeout": "60", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_op_threads": "2", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_punch_hole": "false", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-03-29T14:02:14.903 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_bytes": "104857600", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_ops": "50", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_seek_data_hole": "false", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_sloppy_crc": "false", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_splice": "false", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_split_multiple": "2", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_split_rand_factor": "20", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_update_to": "1000", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-03-29T14:02:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_enable": "true", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_xfs_extsize": "false", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_zfs_snap": "false", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "fio_dir": "/tmp/fio", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "fcbbec2c-93bf-47a4-85ec-620dfd2d0d7a", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_allow_other": "true", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_atomic_o_trunc": "true", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_big_writes": "true", 2023-03-29T14:02:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_debug": "false", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_default_permissions": "false", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_disable_pagecache": "false", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_max_write": "0", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_multithreaded": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_require_active_mds": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_set_user_groups": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_splice_move": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_splice_read": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_splice_write": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_syncfs_on_mksnap": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_use_invalidate_cb": "true", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "gss_target_name": "ceph", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_file": "", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_inject_failure": "0", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_interval": "5", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "host": "smithi017", 2023-03-29T14:02:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_max_size": "1073741824", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_watermark": "0.900000", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "inject_early_sigterm": "false", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "jaeger_agent_port": "6799", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "jaeger_tracing_enable": "false", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_aio": "true", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_align_min_size": "65536", 2023-03-29T14:02:14.907 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_block_align": "true", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_block_size": "4096", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_dio": "true", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_discard": "false", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_force_aio": "false", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_ignore_corruption": "false", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_max_write_bytes": "10485760", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_max_write_entries": "100", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_replay_from": "0", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_high_multiple": "0.000000", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_max_multiple": "0.000000", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_write_header_frequency": "0", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_zero_on_create": "false", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journaler_prefetch_periods": "10", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journaler_prezero_periods": "5", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "journaler_write_head_interval": "15", 2023-03-29T14:02:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "keyfile": "", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_backend": "rocksdb", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_default_stripe_size": "65536", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_fsck_on_mount": "false", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_fsck_on_mount_deep": "true", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_max_bytes": "67108864", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_max_ops": "512", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_nid_prealloc": "1024", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_onode_map_size": "1024", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_sync_submit_transaction": "false", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_sync_transaction": "false", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_block_size": "65536", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_bloom_size": "0", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_cache_size": "536870912", 2023-03-29T14:02:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_compact_on_mount": "false", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_compression": "false", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_log": "", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_log_to_ceph_log": "true", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_max_open_files": "0", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_paranoid": "false", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "leveldb_write_buffer_size": "33554432", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "librados_thread_count": "2", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "lockdep": "false", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "lockdep_force_backtrace": "false", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_coarse_timestamps": "true", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_flush_on_exit": "false", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_graylog_host": "127.0.0.1", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_graylog_port": "12201", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_max_new": "1000", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_max_recent": "10000", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_stderr_prefix": "", 2023-03-29T14:02:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_stop_at_utilization": "0.970000", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": "true", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_graylog": "false", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_journald": "false", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_stderr": "false", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_syslog": "false", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "max_rotating_auth_attempts": "10", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_action_on_write_error": "1", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_alternate_name_max": "8192", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_asio_thread_count": "2", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_export_pin": "true", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_dirs": "true", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_interval": "5", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_size_max": "100000", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_idle_threshold": "0.000000", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_interval": "10", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_max": "-1", 2023-03-29T14:02:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_max_until": "-1", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_merge_size": "50", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_midchunk": "0.300000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_min_rebalance": "0.100000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_min_start": "0.200000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_minchunk": "0.001000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_mode": "0", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_need_max": "1.200000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_need_min": "0.800000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_sample_interval": "3.000000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_bits": "3", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_rd": "25000.000000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_size": "10000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_wr": "10000.000000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_target_decay": "10.000000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_beacon_grace": "15.000000", 2023-03-29T14:02:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_beacon_interval": "4.000000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_beacon_mon_down_grace": "60", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_memory_limit": "4294967296", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_mid": "0.700000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_release_free_interval": "10", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_reservation": "0.050000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_trim_interval": "1", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_trim_threshold": "262144", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_client_delegate_inos_pct": "50", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_client_prealloc_inos": "1000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_connect_bootstrapping": "false", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_damage_table_max_entries": "10000", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_auth_pins": "false", 2023-03-29T14:02:14.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_frag": "false", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_scatterstat": "false", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_subtrees": "false", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_decay_halflife": "5.000000", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_default_dir_hash": "2", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_defer_session_stale": "true", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_deny_all_reconnect": "false", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_keys_per_op": "16384", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_max_commit_size": "10", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_max_entries": "0", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_prefetch": "true", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dirstat_min_interval": "1.000000", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_after_rejoin": "false", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_on_map": "false", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_threshold_file": "0", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_early_reply": "true", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_enable_op_tracker": "true", 2023-03-29T14:02:14.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_enforce_unique_name": "true", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_distributed": "true", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_random": "true", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_forward_all_requests_to_auth": "false", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_health_cache_threshold": "1.500000", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_health_summarize_threshold": "10", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_heartbeat_grace": "15.000000", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_heartbeat_reset_grace": "1000", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_health_dummy": "false", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_migrator_session_race": "false", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_join_fs": "", 2023-03-29T14:02:14.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_journal_format": "1", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_create_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_export_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_import_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_journal_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_journal_expire_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_journal_replay_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_link_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_mdstable_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_openc_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_rename_at": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_events_per_segment": "1024", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_max_events": "-1", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_max_segments": "128", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_pause": "false", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_segment_size": "0", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_skip_corrupt_events": "false", 2023-03-29T14:02:14.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_warn_factor": "2.000000", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_caps_per_client": "1048576", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_completed_flushes": "100000", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_completed_requests": "100000", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_export_size": "20971520", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_file_recover": "32", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_purge_files": "64", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_purge_ops": "8192", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_snaps_per_dir": "100", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_metrics_update_interval": "2", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_min_caps_per_client": "100", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_min_caps_working_set": "10000", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_numa_node": "-1", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-03-29T14:02:14.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_complaint_time": "30.000000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_history_duration": "600", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_history_size": "20", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_log_threshold": "5", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_ping_grace": "15", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_ping_interval": "5", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_max_caps": "30000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_max_decay_threshold": "131072", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_warning_threshold": "262144", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_reconnect_timeout": "45.000000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_replay_interval": "1.000000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_root_ino_gid": "0", 2023-03-29T14:02:14.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_root_ino_uid": "0", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_blocklist_on_evict": "true", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_blocklist_on_timeout": "true", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_shutdown_check": "0", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_skip_ino": "0", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sleep_rank_change": "0.000000", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_snap_max_uid": "4294967294", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_snap_min_uid": "0", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_snap_rstat": "false", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_standby_replay_damaged": "false", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_symlink_recovery": "true", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_task_status_update_interval": "2.000000", 2023-03-29T14:02:14.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_thrash_exports": "0", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_thrash_fragments": "0", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_tick_interval": "5.000000", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_valgrind_exit": "false", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_verify_backtrace": "1", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_verify_scatter": "false", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_wipe_ino_prealloc": "false", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_wipe_sessions": "false", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool_debug": "false", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_debug_omit_block_device_write": "false", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_device_bytes": "1073741824", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_page_set": "false", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_page_size": "65536", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_client_bytes": "134217728", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_client_messages": "512", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_connect_retry_interval": "1.000000", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-03-29T14:02:14.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_disabled_modules": "", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_max_pg_num_change": "128", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mds_bytes": "134217728", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mds_messages": "128", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mon_bytes": "134217728", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mon_messages": "128", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_osd_bytes": "536870912", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_osd_messages": "8192", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_pool": "true", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_service_beacon_grace": "60.000000", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_standby_modules": "true", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_stats_period": "5", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_stats_threshold": "5", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_tick_period": "2", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-03-29T14:02:14.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_accept_timeout_factor": "2.000000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_allow_pool_delete": "true", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_allow_pool_size_one": "true", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_auth_validate_all_caps": "true", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_bytes": "104857600", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_directed_command_retry": "2", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval": "3.000000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_parallel": "3", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_log_interval": "1.000000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_ping_interval": "10.000000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_ping_timeout": "30.000000", 2023-03-29T14:02:14.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_clock_drift_allowed": "1.000000", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_file_level": "debug", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_file": "true", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_graylog": "false", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_journald": "false", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_stderr": "false", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_compact_on_bootstrap": "false", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_compact_on_start": "false", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_compact_on_trim": "true", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_con_tracker_persist_interval": "10", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_con_tracker_score_halflife": "43200", 2023-03-29T14:02:14.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_config_key_max_entry_size": "65536", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cpu_threads": "4", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_crush_min_required_version": "hammer", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_daemon_bytes": "419430400", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data_avail_crit": "5", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data_avail_warn": "5", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data_size_warn": "16106127360", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_block_osdmap_trim": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_dump_json": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_dump_transactions": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_extra_checks": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_require_quincy": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_require_reef": "false", 2023-03-29T14:02:14.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_delta_reset_interval": "10.000000", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_dns_srv_name": "ceph-mon", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_down_mkfs_grace": "120", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_election_default_strategy": "1", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_election_timeout": "5.000000", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_elector_ping_divisor": "2", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_elector_ping_timeout": "2.000000", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_enable_op_tracker": "true", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_fake_pool_delete": "false", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_force_quorum_join": "false", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_globalid_prealloc": "10000", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_detail_to_clog": "false", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_log_update_period": "5", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_max_detail": "50", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_to_clog": "true", 2023-03-29T14:02:14.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_to_clog_interval": "600", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_host": "172.21.15.17", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_host_override": "", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_initial_members": "", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_keyvaluedb": "rocksdb", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_lease": "5.000000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_log_full_interval": "50", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_log_max": "10000", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_log_max_summary": "50", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_log_entries_per_event": "4096", 2023-03-29T14:02:14.926 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_log_epochs": "500", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_mdsmap_epochs": "500", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_mgrmap_epochs": "500", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_osd": "10000", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_pg_per_osd": "10000", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_pool_pg_num": "65536", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mds_force_trim_to": "0", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mds_skip_sanity": "false", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_memory_autotune": "true", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_memory_target": "2147483648", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_beacon_grace": "30", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_digest_period": "5", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_inactive_grace": "60", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_mkfs_grace": "240", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-03-29T14:02:14.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_min_osdmap_epochs": "50", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_complaint_time": "30", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_duration": "600", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_size": "20", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_slow_op_size": "20", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_slow_op_threshold": "10", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_log_threshold": "5", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_auto_mark_in": "false", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_auto_mark_new_in": "true", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_cache_size": "500", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_cache_size_min": "134217728", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_crush_smoke_test": "true", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_destroyed_out_interval": "600", 2023-03-29T14:02:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_down_out_interval": "600", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_force_trim_to": "0", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_full_ratio": "0.950000", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_laggy_halflife": "3600", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_laggy_max_interval": "300", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_laggy_weight": "0.300000", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_max_creating_pgs": "1024", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_max_initial_pgs": "1024", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_min_down_reporters": "2", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_prime_pg_temp": "true", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-03-29T14:02:14.929 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_report_timeout": "900", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_warn_num_repaired": "10", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_warn_op_age": "32.000000", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_interval": "2", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_min": "15", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_stuck_threshold": "60", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_min_objects": "10000", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_min_per_osd": "0", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pool_quota_crit_threshold": "0", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pool_quota_warn_threshold": "0", 2023-03-29T14:02:14.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_probe_timeout": "2.000000", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_max_change": "0.050000", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_max_osds": "4", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_interval": "86400", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_max_keys": "100", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_timeout": "300", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_session_timeout": "300", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_smart_report_timeout": "5", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stat_smooth_intervals": "6", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_pool_min_size": "2", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_pool_size": "4", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-03-29T14:02:14.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_subscribe_interval": "86400.000000", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_debug": "false", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_max_payload_keys": "2000", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_max_payload_size": "1048576", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_provider_kill_at": "0", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_requester_kill_at": "0", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_timeout": "60.000000", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_target_pg_per_osd": "100", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_tick_interval": "5", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_timecheck_interval": "300.000000", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_use_min_delay_socket": "false", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_older_version_delay": "604800", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_filestore_osds": "false", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-03-29T14:02:14.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_misplaced": "false", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_older_version": "true", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_pool_no_app": "true", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_too_few_osds": "false", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": "", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-03-29T14:02:14.933 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_tracing_enabled": "false", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_op_threads": "3", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_buffer_size": "131072", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_cm": "false", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_device_name": "", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_dscp": "96", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_gid_idx": "0", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_local_gid": "", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_polling_us": "1000", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_port_num": "1", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_roce_ver": "1", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_send_buffers": "1024", 2023-03-29T14:02:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_sl": "3", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_support_srq": "true", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_type": "ib", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_reap_threshold": "5", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_before_connect": "false", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_ipv4": "true", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_ipv6": "false", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_msgr1": "true", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_msgr2": "true", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_port_max": "7568", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_port_min": "6800", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_prefer_ipv4": "false", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_retry_count": "3", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_retry_delay": "5", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_client": "false", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_mds": "false", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_mgr": "false", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_mon": "false", 2023-03-29T14:02:14.935 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_osd": "false", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_client_mode": "crc", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_cluster_mode": "crc", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_cluster_type": "", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_compress_secure": "false", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_connection_idle_timeout": "900", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_connection_ready_timeout": "10", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_crc_data": "true", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_crc_header": "true", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_bad_msg": "false", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_bug": "true", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_old_message": "true", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_skipped_message": "false", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_unhandled_msg": "false", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_coremask": "0xF", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_devs_allowlist": "", 2023-03-29T14:02:14.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_enable_tso": "true", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_hugepages": "", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_hw_flow_control": "true", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_lro": "true", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_memory_channel": "4", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_pmd": "", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_port_id": "0", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dump_corrupt_message_level": "1", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dump_on_send": "false", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_initial_backoff": "0.200000", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_delay_max": "1.000000", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_delay_probability": "0.000000", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_delay_type": "", 2023-03-29T14:02:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_internal_delays": "0.000000", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_network_congestion": "0", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_socket_failures": "0", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_learn_addr_from_peer": "true", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_max_accept_failures": "4", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_max_backoff": "15.000000", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_mon_client_mode": "crc", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_mon_cluster_mode": "crc", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_mon_service_mode": "crc", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_osd_compress_min_size": "1024", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_osd_compress_mode": "none", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_osd_compression_algorithm": "snappy", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_pq_min_cost": "65536", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_public_type": "", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_service_mode": "crc", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_listen_backlog": "512", 2023-03-29T14:02:14.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_nodelay": "true", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_rcvbuf": "0", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_type": "async+posix", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "no_config_file": "false", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_completion_locks_per_session": "32", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_debug_inject_relock_delay": "false", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_inflight_op_bytes": "104857600", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_inflight_ops": "1024", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_inject_no_watch_ping": "false", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_tick_interval": "5.000000", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_timeout": "10.000000", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "objectstore_blackhole": "false", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "openssl_engine_opts": "", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_delay_time": "5.000000", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_hist_halflife": "1000", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_max_low_ops": "2", 2023-03-29T14:02:14.939 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_max_ops": "4", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_quantize_effort": "0.100000", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_slop": "0.020000", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_allow_recovery_below_min_size": "true", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_asio_thread_count": "2", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_async_recovery_min_cost": "100", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_auto_mark_unfound_lost": "false", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backfill_retry_interval": "30.000000", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backfill_scan_max": "512", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backfill_scan_min": "64", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backoff_on_degraded": "false", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backoff_on_peering": "false", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backoff_on_unfound": "true", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_beacon_report_interval": "300", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_duration": "30", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-03-29T14:02:14.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_max_block_size": "67108864", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_small_size_max_iops": "100", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_blkin_trace_all": "false", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_blocked_scrub_grace_period": "120", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_check_for_log_corruption": "false", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_update_on_start": "true", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_message_cap": "256", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_message_size_cap": "524288000", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_op_priority": "63", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_watch_timeout": "30", 2023-03-29T14:02:14.941 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_command_max_records": "256", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_compact_on_start": "false", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_chooseleaf_type": "0", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_initial_weight": "-1.000000", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_update_on_start": "true", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_update_weight_set": "true", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_drop_ping_duration": "0", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_feed_pullee": "-1", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_misdirected_ops": "false", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_no_acting_change": "false", 2023-03-29T14:02:14.942 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_no_purge_strays": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_op_order": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_pg_log_writeout": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_pretend_recovery_active": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_shutdown": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_cached_snaps": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_missing_on_start": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_snaps": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_stray_on_activate": "false", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_interval": "604800.000000", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_keys": "1024", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2023-03-29T14:02:14.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_stride": "524288", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_default_data_pool_replay_window": "5", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_default_notify_timeout": "30", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep": "0.000000", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep_hdd": "5.000000", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep_hybrid": "1.000000", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep_ssd": "1.000000", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_discard_disconnected_ops": "true", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_enable_op_tracker": "true", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_extblkdev_plugins": "vdo", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_failsafe_full_ratio": "0.970000", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_fail_on_connection_refused": "true", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_info": "true", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_shutdown": "true", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_shutdown_notify_mon": "true", 2023-03-29T14:02:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_shutdown_timeout": "15", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_find_best_info_ignore_history_les": "false", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_force_auth_primary_missing_objects": "100", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_function_tracing": "false", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_grace": "20", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_interval": "6", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_min_peers": "10", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_min_size": "2000", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_stale": "600", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_hit_set_max_size": "100000", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_hit_set_min_size": "1000", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_hit_set_namespace": ".ceph-internal", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_ignore_stale_divergent_priors": "false", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_inject_failure_on_pg_removal": "false", 2023-03-29T14:02:14.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_journal_flush_on_shutdown": "true", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_journal_size": "5120", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_kill_backfill_at": "0", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_loop_before_reset_tphandle": "64", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_cache_size": "50", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_dedup": "true", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_message_max": "40", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_message_max_bytes": "10485760", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_share_max_epochs": "40", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_attr_name_len": "100", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_attr_size": "0", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_backfills": "1", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_markdown_count": "5", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_markdown_period": "600", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_object_name_len": "2048", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_object_namespace_len": "256", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_object_size": "134217728", 2023-03-29T14:02:14.946 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_omap_entries_per_request": "1024", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pg_blocked_by": "16", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pg_log_entries": "10000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pgls": "1024", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_push_cost": "8388608", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_push_objects": "10", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_scrubs": "1", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_trimming_pgs": "2", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_write_op_reply_len": "64", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_write_size": "90", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_cost_per_byte_usec": "0.000000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_cost_per_byte_usec_hdd": "2.600000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_cost_per_byte_usec_ssd": "0.011000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_cost_per_io_usec": "0.000000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_cost_per_io_usec_hdd": "11400.000000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_cost_per_io_usec_ssd": "50.000000", 2023-03-29T14:02:14.947 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_override_recovery_settings": "false", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_profile": "high_client_ops", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_client_lim": "999999", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_client_res": "1", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_client_wgt": "1", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_skip_benchmark": "false", 2023-03-29T14:02:14.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_base": "805306368", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_cache_min": "134217728", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_cache_resize_interval": "1.000000", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_expected_fragmentation": "0.150000", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_target": "4294967296", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_target_autotune": "false", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_min_pg_log_entries": "250", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_min_recovery_priority": "0", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_heartbeat_interval": "30", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_report_interval": "5", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_report_max_in_flight": "2", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_shutdown_timeout": "5.000000", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_num_cache_shards": "32", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_num_op_tracker_shard": "32", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_numa_auto_affinity": "true", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_numa_node": "-1", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_numa_prefer_iface": "true", 2023-03-29T14:02:14.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_object_clean_region_max_num_intervals": "10", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objecter_finishers": "1", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objectstore": "bluestore", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objectstore_fuse": "false", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objectstore_tracing": "false", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_complaint_time": "30.000000", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_duration": "600", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_size": "20", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_slow_op_size": "20", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_log_threshold": "5", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_shards": "0", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_shards_hdd": "5", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_shards_ssd": "8", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_threads_per_shard": "0", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_pq_min_cost": "65536", 2023-03-29T14:02:14.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_queue": "mclock_scheduler", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_queue_cut_off": "high", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_thread_suicide_timeout": "150", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_thread_timeout": "15", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_open_classes_on_start": "true", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_os_flags": "0", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_peering_op_priority": "255", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_delete_cost": "1048576", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_delete_priority": "5", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_log_dups_tracked": "3000", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_log_trim_max": "10000", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_log_trim_min": "100", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_object_context_cache_count": "64", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_stat_report_interval_max": "500", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_min_evict_age": "0", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_min_flush_age": "0", 2023-03-29T14:02:14.951 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_crimson": "true", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_crush_rule": "-1", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_ec_fast_read": "false", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_bulk": "false", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_hashpspool": "true", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_nodelete": "false", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_nopgchange": "false", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_nosizechange": "false", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flags": "0", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_min_size": "2", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_pg_num": "32", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_pgp_num": "0", 2023-03-29T14:02:14.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_size": "3", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_type": "replicated", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_use_gmt_hitset": "true", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_push_per_object_cost": "1000", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_read_ec_check_for_errors": "false", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recover_clone_overlap": "true", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recover_clone_overlap_limit": "10", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_cost": "20971520", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_delay_start": "0.000000", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_active": "0", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_active_hdd": "3", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_active_ssd": "10", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_chunk": "8388608", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_single_start": "1", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_op_priority": "3", 2023-03-29T14:02:14.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_op_warn_multiple": "16", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_priority": "5", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_retry_interval": "30.000000", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep": "0.000000", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep_hdd": "0.100000", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep_ssd": "0.000000", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_repair_during_recovery": "false", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_requested_scrub_priority": "120", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_rollback_to_cluster_snap": "", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_auto_repair": "false", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_auto_repair_num_errors": "5", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_backoff_ratio": "0.660000", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_begin_hour": "0", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_begin_week_day": "0", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_chunk_max": "25", 2023-03-29T14:02:14.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_chunk_min": "5", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_cost": "52428800", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_during_recovery": "false", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_end_hour": "0", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_end_week_day": "0", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_extended_sleep": "0.000000", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_invalid_stats": "true", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_load_threshold": "0.500000", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_max_interval": "604800.000000", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_max_preemptions": "5", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_min_interval": "86400.000000", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_priority": "5", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_reservation_timeout": "5000", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_sleep": "0.000000", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_slow_reservation_response": "2200", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_shallow_scrub_chunk_max": "100", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_shallow_scrub_chunk_min": "50", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_shutdown_pgref_assert": "false", 2023-03-29T14:02:14.955 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_skip_data_digest": "false", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_smart_report_timeout": "5", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_cost": "1048576", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_priority": "5", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep": "0.000000", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_stats_update_period_not_scrubbing": "120", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_stats_update_period_scrubbing": "15", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_target_transaction_size": "30", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_count": "4", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2023-03-29T14:02:14.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_mode": "writeback", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_promote_max_objects_sec": "25", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tracing": "false", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_use_stale_snap": "false", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osdc_blkin_trace_all": "false", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_kill_at": "0", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_max_join_drift": "10", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_min": "500", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_min_wait": "0.050000", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_propose_interval": "1.000000", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_service_trim_max": "500", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_service_trim_max_multiplier": "20", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_service_trim_min": "10", 2023-03-29T14:02:14.957 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_stash_full_interval": "25", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_trim_max": "500", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_trim_min": "250", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "perf": "true", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin_crypto_accelerator": "crypto_isal", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin_dir": "/usr/lib64/ceph", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "-", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrv": "", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "public_bind_addr": "-", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "public_network": "", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "public_network_interface": "", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "qat_compressor_enabled": "false", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "qat_compressor_session_max_number": "256", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "rados_mon_op_timeout": "0", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "rados_osd_op_timeout": "0", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "rados_tracing": "false", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_atime_update_interval": "60", 2023-03-29T14:02:14.958 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_balance_parent_reads": "false", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_balance_snap_reads": "false", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_blkin_trace_all": "false", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_blocklist_expire_seconds": "0", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_blocklist_on_break_lock": "true", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache": "true", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_block_writes_upfront": "false", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_max_dirty": "25165824", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_max_dirty_age": "1.000000", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_max_dirty_object": "0", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_policy": "writearound", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_size": "33554432", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_target_dirty": "16777216", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_writethrough_until_flush": "true", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_clone_copy_on_read": "false", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_compression_hint": "none", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_concurrent_management_ops": "10", 2023-03-29T14:02:14.959 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_config_pool_override_update_timestamp": "0", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_clone_format": "auto", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_data_pool": "", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_features": "61", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_format": "2", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_map_options": "", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_order": "22", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_pool": "rbd", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_stripe_count": "0", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_stripe_unit": "0", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_disable_zero_copy_writes": "true", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_discard_granularity_bytes": "65536", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_discard_on_zeroed_write_same": "true", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_enable_alloc_hint": "true", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_io_scheduler": "simple", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_commit_age": "5.000000", 2023-03-29T14:02:14.960 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_max_payload_bytes": "16384", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_flush_age": "0.000000", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_flush_bytes": "1048576", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_flush_interval": "0", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_order": "24", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_pool": "", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_splay_width": "4", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_localize_parent_reads": "false", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_localize_snap_reads": "false", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2023-03-29T14:02:14.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_type": "simple", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_state_check_interval": "30", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_autotune": "true", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_base": "805306368", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_cache_min": "134217728", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_target": "4294967296", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_perf_stats_prio": "5", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2023-03-29T14:02:14.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_delete_delay": "0", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_replay_delay": "0", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_move_to_trash_on_remove": "false", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mtime_update_interval": "60", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_non_blocking_aio": "true", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_op_thread_timeout": "60", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_op_threads": "1", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_parent_cache_enabled": "false", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_persistent_cache_mode": "disabled", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_persistent_cache_path": "/tmp", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_persistent_cache_size": "1073741824", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_plugins": "", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_bps_burst": "0", 2023-03-29T14:02:14.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_bps_burst_seconds": "1", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_bps_limit": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_exclude_ops": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_iops_burst": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_iops_burst_seconds": "1", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_iops_limit": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_bps_burst": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_bps_limit": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_iops_burst": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_iops_limit": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_schedule_tick_min": "50", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_bps_burst": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_bps_limit": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_iops_burst": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_iops_limit": "0", 2023-03-29T14:02:14.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_quiesce_notification_attempts": "10", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_read_from_replica_policy": "default", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_readahead_max_bytes": "524288", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_readahead_trigger_requests": "10", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_request_timed_out_seconds": "30", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_skip_partial_discard": "true", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_tracing": "false", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_validate_names": "true", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_validate_pool": "true", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "restapi_base_url": "", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "restapi_log_level": "", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_acl_grants_max_num": "100", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_admin_entry": "admin", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_backend_store": "rados", 2023-03-29T14:02:14.965 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_barbican_url": "", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_beast_enable_async": "true", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_default_quota_max_size": "-1", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_index_max_aio": "128", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_quota_cache_size": "10000", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_quota_ttl": "600", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_sync_spawn_window": "20", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cache_enabled": "true", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cache_expiry_interval": "900", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cache_lru_size": "10000", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_config_store": "rados", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_content_length_compat": "false", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_copy_obj_progress": "true", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cors_rules_max_num": "100", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cross_domain_policy": "", 2023-03-29T14:02:14.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_default_encryption_key": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_addr": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_ca_path": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_client_cert": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_client_key": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_kms_key_template": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_password": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_username": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_require_ssl": "true", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_backend": "vault", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2023-03-29T14:02:14.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_suppress_logs": "true", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_addr": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_auth": "token", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_namespace": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_prefix": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_secret_engine": "transit", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_ssl_cacert": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_token_file": "", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_verify_ssl": "true", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_buffersize": "524288", 2023-03-29T14:02:14.968 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_low_speed_limit": "1024", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_low_speed_time": "300", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_tcp_keepalive": "0", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_wait_timeout_ms": "1000", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_fadvise": "4", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_libaio_aio_num": "64", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_libaio_aio_threads": "20", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_changes_size": "1000", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_num_shards": "128", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_obj_prefix": "data_log", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_window": "30", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_notify_interval_msec": "0", 2023-03-29T14:02:14.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_sync_poll_interval": "20", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_sync_spawn_window": "20", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_data_log_backing": "fifo", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_realm_info_oid": "default.realm", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_region_info_oid": "default.region", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_zone_info_oid": "default.zone", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_defer_to_bucket_acls": "", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_delete_multi_obj_max_num": "1000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_admin_lim": "0.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_admin_res": "100.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_auth_lim": "0.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_auth_res": "200.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_data_lim": "0.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_data_res": "500.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_data_wgt": "500.000000", 2023-03-29T14:02:14.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_metadata_res": "500.000000", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dns_name": "", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dns_s3website_name": "", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dynamic_resharding": "true", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_gc_threads": "true", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_lc_threads": "true", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_ops_log": "false", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_quota_threads": "true", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_static_website": "false", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_usage_log": "false", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enforce_swift_acls": "true", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_exit_timeout_secs": "120", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_expose_bucket": "false", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_extended_http_attrs": "", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_filter": "none", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2023-03-29T14:02:14.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_frontends": "beast port=7480", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_concurrent_io": "10", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_deferred": "50", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_objs": "32", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_queue_size": "134213632", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_trim_chunk": "16", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_obj_min_wait": "7200", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_processor_max_time": "3600", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_processor_period": "3600", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_get_obj_max_req_size": "4194304", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_get_obj_window_size": "16777216", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_healthcheck_disabling_path": "", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ignore_get_invalid_range": "false", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_init_timeout": "300", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_accepted_admin_roles": "", 2023-03-29T14:02:14.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_accepted_reader_roles": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_domain": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_password": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_password_path": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_project": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_tenant": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_token": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_token_path": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_user": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_api_version": "2", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_domain": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_password": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_project": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_tenant": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_user": "", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_implicit_tenants": "false", 2023-03-29T14:02:14.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_service_token_enabled": "false", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_token_cache_size": "10000", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_url": "", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_verify_ssl": "true", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_debug_interval": "-1", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_lock_max_time": "90", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_objs": "32", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_rules": "1000", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_worker": "3", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_wp_worker": "3", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_thread_delay": "0", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_dnattr": "uid", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_searchfilter": "", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_uri": "ldaps://", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2023-03-29T14:02:14.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_list_bucket_min_readahead": "1000", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_list_buckets_max_chunk": "1000", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_http_headers": "", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_nonexistent_bucket": "false", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_object_name_utc": "false", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_attr_name_len": "0", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_attr_size": "0", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_attrs_num_in_req": "0", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_chunk_size": "4194304", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_concurrent_requests": "1024", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_dynamic_shards": "1999", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_listing_results": "1000", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_notify_retries": "3", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_objs_per_shard": "100000", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_put_param_size": "1048576", 2023-03-29T14:02:14.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_put_size": "5368709120", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_slo_entries": "1000", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_md_log_max_shards": "64", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_md_notify_interval_msec": "200", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_meta_sync_poll_interval": "20", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_meta_sync_spawn_window": "20", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_mime_types_file": "/etc/mime.types", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_mp_lock_max_time": "600", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_multi_obj_del_max_aio": "16", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_multipart_min_part_size": "5242880", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_multipart_part_upload_limit": "10000", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_fhcache_partitions": "3", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_fhcache_size": "2017", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_frontends": "rgw-nfs", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_lru_lanes": "5", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_max_gc": "300", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_namespace_expire_secs": "300", 2023-03-29T14:02:14.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_gc_threads": "false", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_lc_threads": "false", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_quota_threads": "false", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_sync_thread": "false", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_s3_fast_attrs": "false", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_write_completion_interval_s": "10", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_num_async_rados_threads": "32", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_num_control_oids": "8", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_numa_node": "-1", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_obj_stripe_size": "4194304", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_obj_tombstone_cache_size": "1000", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_objexp_chunk_size": "100", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_objexp_gc_interval": "600", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_objexp_hints_num_shards": "127", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_olh_pending_timeout_sec": "3600", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_op_thread_suicide_timeout": "0", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_op_thread_timeout": "600", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_op_tracing": "false", 2023-03-29T14:02:14.977 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_opa_token": "", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_opa_url": "", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_opa_verify_ssl": "true", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_data_backlog": "5242880", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_rados": "false", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_socket_path": "", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_override_bucket_index_max_shards": "0", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_push_interval": "2.000000", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_push_interval_max": "30.000000", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_root_pool": ".rgw.root", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_policy_reject_invalid_principals": "true", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_print_continue": "true", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_print_prohibited_content_length": "false", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_put_obj_max_window_size": "67108864", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_put_obj_min_window_size": "16777216", 2023-03-29T14:02:14.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rados_pool_recovery_priority": "5", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rados_tracing": "false", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_realm": "", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_realm_id": "", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_realm_root_pool": ".rgw.root", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_region": "", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_region_root_pool": ".rgw.root", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_relaxed_region_enforcement": "false", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_relaxed_s3_bucket_names": "false", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_request_uri": "", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_batch_size": "64", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_bucket_lock_duration": "360", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_max_aio": "128", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_num_logs": "16", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_thread_interval": "600", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_resolve_cname": "false", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rest_getusage_op_compat": "false", 2023-03-29T14:02:14.979 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_run_sync_thread": "true", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_order": "sts, external, local", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_keystone": "false", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_ldap": "false", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_rados": "true", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_sts": "false", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_client_max_sig_ver": "-1", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_success_create_obj_status": "0", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_safe_max_objects_per_shard": "102400", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_scheduler_type": "throttler", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_script_uri": "", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_service_provider_name": "", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_shard_warning_threshold": "90.000000", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_client_id": "", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_client_secret": "", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_entry": "sts", 2023-03-29T14:02:14.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_key": "sts", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_max_session_duration": "43200", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_min_session_duration": "900", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_token_introspection_url": "", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_account_in_url": "false", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_auth_entry": "auth", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_auth_url": "", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_custom_header": "", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_enforce_content_length": "false", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_need_stats": "true", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_tenant_name": "", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_token_expiration": "86400", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_url": "", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_url_prefix": "swift", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_versioning_enabled": "false", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2023-03-29T14:02:14.981 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_lease_period": "120", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_interval": "1200", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_max_buckets": "16", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_obj_etag_verify": "false", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_trace_history_size": "4096", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_trace_per_node_log_size": "32", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_thread_pool_size": "512", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_comment": "", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_createby": "", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_encoding": "", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_flag": "false", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_max_size": "5368709120", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_origin": "", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_sha_unit": "524288", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_tracker": "", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_trust_forwarded_https": "false", 2023-03-29T14:02:14.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_log_flush_threshold": "1024", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_log_tick_interval": "30", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_max_shards": "32", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_max_user_shards": "1", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_use_opa_authz": "false", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_default_quota_max_objects": "-1", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_default_quota_max_size": "-1", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_max_buckets": "1000", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_policies_max_num": "100", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_sync_idle_users": "false", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_sync_interval": "86400", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_sync_wait_time": "86400", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_unique_email": "true", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_verify_ssl": "true", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_website_routing_rules_max_num": "50", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zone": "", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zone_id": "", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zone_root_pool": ".rgw.root", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zonegroup": "", 2023-03-29T14:02:14.983 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zonegroup_id": "", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rgwlc_auto_session_clear": "true", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rgwlc_skip_bucket_step": "false", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_block_size": "4096", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_bloom_bits_per_key": "20", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_row_ratio": "0.000000", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_shard_bits": "4", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_size": "536870912", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_type": "binned_lru", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cf_compact_on_deletion": "true", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_collect_compaction_stats": "false", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_collect_extended_stats": "false", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_collect_memory_stats": "false", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_delete_range_threshold": "1048576", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_index_type": "binary_search", 2023-03-29T14:02:14.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_log_to_ceph_log": "true", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_metadata_block_size": "4096", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_partition_filters": "false", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_perf": "false", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "rotating_keys_bootstrap_timeout": "30", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "rotating_keys_renewal_timeout": "10", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "run_dir": "/var/run/ceph", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_block_create": "true", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_cache_lru_size": "67108864", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_cbjournal_size": "5368709120", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_default_max_object_size": "16777216", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_default_object_metadata_reservation": "16777216", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_device_size": "53687091200", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_batch_capacity": "16", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_batch_flush_size": "16777216", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_iodepth_limit": "5", 2023-03-29T14:02:14.985 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_main_device_type": "SSD", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_max_concurrent_transactions": "8", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_obj_data_write_amplification": "1.250000", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_segment_size": "67108864", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "set_keepcaps": "false", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "setgroup": "", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "setuser": "", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "setuser_match_path": "", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_misplaced_ratio": "0.050000", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "thp": "false", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "threadpool_default_timeout": "60", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "threadpool_empty_queue_max_wait": "2", 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout: "throttler_perf_counter": "true" 2023-03-29T14:02:14.986 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-03-29T14:02:14.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2023-03-29T14:02:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.117+0000 7f5e4f768700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.122+0000 7f5e4f768700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.136 INFO:teuthology.orchestra.run.smithi017.stdout:{} 2023-03-29T14:02:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:mon_osd_down_out_interval = '0' (not observed, change may require restart) 2023-03-29T14:02:15.149 DEBUG:teuthology.orchestra.run.smithi017:> type ceph-objectstore-tool 2023-03-29T14:02:15.166 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-03-29T14:02:15.166 DEBUG:teuthology.orchestra.run.smithi089:> type ceph-objectstore-tool 2023-03-29T14:02:15.187 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-03-29T14:02:15.187 INFO:teuthology.run_tasks:Running task rados... 2023-03-29T14:02:15.199 INFO:tasks.rados:Beginning rados... 2023-03-29T14:02:15.200 DEBUG:teuthology.run_tasks:Unwinding manager rados 2023-03-29T14:02:15.219 INFO:tasks.rados:joining rados 2023-03-29T14:02:15.220 INFO:tasks.thrashosds.thrasher:starting do_thrash 2023-03-29T14:02:15.220 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:15.220 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:15.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:15.220 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2023-03-29T14:02:15.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:15.221 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2023-03-29T14:02:15.221 INFO:tasks.rados:starting run 0 out of 1 2023-03-29T14:02:15.221 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2023-03-29T14:02:15.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2023-03-29T14:02:15.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.357+0000 7f8b98394700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.363+0000 7f8b98394700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.365+0000 7f9011fc0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.365+0000 7f757467e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.376+0000 7f757467e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:15.377+0000 7f9011fc0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:15.570 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:15.584 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-03-29T14:02:14.095189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":20112,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":19712,"kb_avail":377467248,"statfs":{"total":386547056640,"available":386526461952,"internally_reserved":0,"allocated":327680,"data_stored":1190076,"data_compressed":52313,"data_compressed_allocated":69632,"data_compressed_original":1089536,"omap_allocated":0,"internal_metadata":20185088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262144,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6,"num_write":28,"num_write_kb":520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.486375"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.274938+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:10.274938+0000","last_peered":"2023-03-29T14:02:10.274938+0000","last_clean":"2023-03-29T14:02:10.274938+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:10.274938+0000","last_undegraded":"2023-03-29T14:02:10.274938+0000","last_fullsized":"2023-03-29T14:02:10.274938+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.353942+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:10.353942+0000","last_peered":"2023-03-29T14:02:10.353942+0000","last_clean":"2023-03-29T14:02:10.353942+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:10.353942+0000","last_undegraded":"2023-03-29T14:02:10.353942+0000","last_fullsized":"2023-03-29T14:02:10.353942+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.374276+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:10.374276+0000","last_peered":"2023-03-29T14:02:10.374276+0000","last_clean":"2023-03-29T14:02:10.374276+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:10.374276+0000","last_undegraded":"2023-03-29T14:02:10.374276+0000","last_fullsized":"2023-03-29T14:02:10.374276+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.367291+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:10.367291+0000","last_peered":"2023-03-29T14:02:10.367291+0000","last_clean":"2023-03-29T14:02:10.367291+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:10.367291+0000","last_undegraded":"2023-03-29T14:02:10.367291+0000","last_fullsized":"2023-03-29T14:02:10.367291+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'50","reported_seq":131,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.573177+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:11.573177+0000","last_peered":"2023-03-29T14:02:11.573177+0000","last_clean":"2023-03-29T14:02:11.573177+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:11.573177+0000","last_undegraded":"2023-03-29T14:02:11.573177+0000","last_fullsized":"2023-03-29T14:02:11.573177+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.316305+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:10.316305+0000","last_peered":"2023-03-29T14:02:10.316305+0000","last_clean":"2023-03-29T14:02:10.316305+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:10.316305+0000","last_undegraded":"2023-03-29T14:02:10.316305+0000","last_fullsized":"2023-03-29T14:02:10.316305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.333140+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:10.333140+0000","last_peered":"2023-03-29T14:02:10.333140+0000","last_clean":"2023-03-29T14:02:10.333140+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:10.333140+0000","last_undegraded":"2023-03-29T14:02:10.333140+0000","last_fullsized":"2023-03-29T14:02:10.333140+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.375236+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:10.375236+0000","last_peered":"2023-03-29T14:02:10.375236+0000","last_clean":"2023-03-29T14:02:10.375236+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:10.375236+0000","last_undegraded":"2023-03-29T14:02:10.375236+0000","last_fullsized":"2023-03-29T14:02:10.375236+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-03-29T14:02:10.410007+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:10.410007+0000","last_peered":"2023-03-29T14:02:10.410007+0000","last_clean":"2023-03-29T14:02:10.410007+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:10.410007+0000","last_undegraded":"2023-03-29T14:02:10.410007+0000","last_fullsized":"2023-03-29T14:02:10.410007+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":918560,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":74,"num_write":166,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":3,"acting":3,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4860,"kb_used_data":52,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366980,"statfs":{"total":96636764160,"available":96631787520,"internally_reserved":0,"allocated":53248,"data_stored":17281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5176,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366664,"statfs":{"total":96636764160,"available":96631463936,"internally_reserved":0,"allocated":98304,"data_stored":479189,"data_compressed":21699,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5036,"kb_used_data":84,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":86016,"data_stored":345489,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5040,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":90112,"data_stored":348117,"data_compressed":15307,"data_compressed_allocated":20480,"data_compressed_original":319488,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:16.200 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'unique_pool_0' created 2023-03-29T14:02:16.214 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2023-03-29T14:02:16.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:16.352+0000 7f256fa62700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:16.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:16.364+0000 7f256fa62700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:17.202 INFO:teuthology.orchestra.run.smithi017.stderr:enabled application 'rados' on pool 'unique_pool_0' 2023-03-29T14:02:17.477 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:17.610+0000 7f27fd4c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:17.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:17.615+0000 7f27fd4c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:18.217 DEBUG:teuthology.orchestra.run.smithi017:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 0 --op copy_from 0 --op write_excl 50 --pool unique_pool_0 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:adding op weight read -> 100 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:adding op weight write -> 50 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:adding op weight delete -> 50 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:adding op weight snap_create -> 50 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:adding op weight snap_remove -> 50 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:adding op weight write_excl -> 50 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:ceph version 18.0.0-3104-g6c07bf89 (6c07bf89555b2fa3413019059913f7a1fb12fde5) reef (dev) 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout:Configuration: 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout: Number of operations: 4000 2023-03-29T14:02:18.255 INFO:tasks.rados.rados.0.smithi017.stdout: Number of objects: 50 2023-03-29T14:02:18.256 INFO:tasks.rados.rados.0.smithi017.stdout: Max in flight operations: 16 2023-03-29T14:02:18.256 INFO:tasks.rados.rados.0.smithi017.stdout: Object size (in bytes): 4000000 2023-03-29T14:02:18.256 INFO:tasks.rados.rados.0.smithi017.stdout: Write stride min: 400000 2023-03-29T14:02:18.256 INFO:tasks.rados.rados.0.smithi017.stdout: Write stride max: 800000 2023-03-29T14:02:19.205 INFO:tasks.rados.rados.0.smithi017.stdout:1: write initial oid 1 2023-03-29T14:02:19.205 INFO:tasks.rados.rados.0.smithi017.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2023-03-29T14:02:19.210 INFO:tasks.rados.rados.0.smithi017.stdout:1: writing smithi017107884-1 from 416807 to 892056 tid 1 2023-03-29T14:02:19.216 INFO:tasks.rados.rados.0.smithi017.stdout:1: writing smithi017107884-1 from 1542129 to 2016807 tid 2 2023-03-29T14:02:19.217 INFO:tasks.rados.rados.0.smithi017.stdout:2: write initial oid 2 2023-03-29T14:02:19.217 INFO:tasks.rados.rados.0.smithi017.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2023-03-29T14:02:19.221 INFO:tasks.rados.rados.0.smithi017.stdout:2: writing smithi017107884-2 from 416807 to 892056 tid 1 2023-03-29T14:02:19.228 INFO:tasks.rados.rados.0.smithi017.stdout:2: writing smithi017107884-2 from 1542129 to 2016807 tid 2 2023-03-29T14:02:19.228 INFO:tasks.rados.rados.0.smithi017.stdout:3: write initial oid 3 2023-03-29T14:02:19.228 INFO:tasks.rados.rados.0.smithi017.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2023-03-29T14:02:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:3: writing smithi017107884-3 from 433614 to 984112 tid 1 2023-03-29T14:02:19.240 INFO:tasks.rados.rados.0.smithi017.stdout:3: writing smithi017107884-3 from 1600611 to 2033614 tid 2 2023-03-29T14:02:19.240 INFO:tasks.rados.rados.0.smithi017.stdout:4: write initial oid 4 2023-03-29T14:02:19.240 INFO:tasks.rados.rados.0.smithi017.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2023-03-29T14:02:19.246 INFO:tasks.rados.rados.0.smithi017.stdout:4: writing smithi017107884-4 from 450421 to 1076168 tid 1 2023-03-29T14:02:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:4: writing smithi017107884-4 from 1659093 to 2050421 tid 2 2023-03-29T14:02:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:5: write initial oid 5 2023-03-29T14:02:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2023-03-29T14:02:19.259 INFO:tasks.rados.rados.0.smithi017.stdout:5: writing smithi017107884-5 from 467228 to 1168224 tid 1 2023-03-29T14:02:19.264 INFO:tasks.rados.rados.0.smithi017.stdout:5: writing smithi017107884-5 from 1717575 to 2067228 tid 2 2023-03-29T14:02:19.264 INFO:tasks.rados.rados.0.smithi017.stdout:6: write initial oid 6 2023-03-29T14:02:19.264 INFO:tasks.rados.rados.0.smithi017.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2023-03-29T14:02:19.271 INFO:tasks.rados.rados.0.smithi017.stdout:6: writing smithi017107884-6 from 484035 to 1260280 tid 1 2023-03-29T14:02:19.276 INFO:tasks.rados.rados.0.smithi017.stdout:6: writing smithi017107884-6 from 2059704 to 2084035 tid 2 2023-03-29T14:02:19.276 INFO:tasks.rados.rados.0.smithi017.stdout:7: write initial oid 7 2023-03-29T14:02:19.276 INFO:tasks.rados.rados.0.smithi017.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2023-03-29T14:02:19.281 INFO:tasks.rados.rados.0.smithi017.stdout:7: writing smithi017107884-7 from 500842 to 952336 tid 1 2023-03-29T14:02:19.288 INFO:tasks.rados.rados.0.smithi017.stdout:7: writing smithi017107884-7 from 1718186 to 2100842 tid 2 2023-03-29T14:02:19.288 INFO:tasks.rados.rados.0.smithi017.stdout:8: write initial oid 8 2023-03-29T14:02:19.288 INFO:tasks.rados.rados.0.smithi017.stdout:2: finishing write tid 1 to smithi017107884-2 2023-03-29T14:02:19.288 INFO:tasks.rados.rados.0.smithi017.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2023-03-29T14:02:19.294 INFO:tasks.rados.rados.0.smithi017.stdout:8: writing smithi017107884-8 from 517649 to 1044392 tid 1 2023-03-29T14:02:19.301 INFO:tasks.rados.rados.0.smithi017.stdout:8: writing smithi017107884-8 from 1776668 to 2117649 tid 2 2023-03-29T14:02:19.301 INFO:tasks.rados.rados.0.smithi017.stdout:9: write initial oid 9 2023-03-29T14:02:19.301 INFO:tasks.rados.rados.0.smithi017.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2023-03-29T14:02:19.308 INFO:tasks.rados.rados.0.smithi017.stdout:9: writing smithi017107884-9 from 534456 to 1252801 tid 1 2023-03-29T14:02:19.313 INFO:tasks.rados.rados.0.smithi017.stdout:9: writing smithi017107884-9 from 1951503 to 2134456 tid 2 2023-03-29T14:02:19.313 INFO:tasks.rados.rados.0.smithi017.stdout:10: write initial oid 10 2023-03-29T14:02:19.313 INFO:tasks.rados.rados.0.smithi017.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2023-03-29T14:02:19.321 INFO:tasks.rados.rados.0.smithi017.stdout:10: writing smithi017107884-10 from 551263 to 1344857 tid 1 2023-03-29T14:02:19.326 INFO:tasks.rados.rados.0.smithi017.stdout:10: writing smithi017107884-10 from 1893632 to 2151263 tid 2 2023-03-29T14:02:19.326 INFO:tasks.rados.rados.0.smithi017.stdout:11: write initial oid 11 2023-03-29T14:02:19.326 INFO:tasks.rados.rados.0.smithi017.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2023-03-29T14:02:19.331 INFO:tasks.rados.rados.0.smithi017.stdout:11: writing smithi017107884-11 from 568070 to 1036913 tid 1 2023-03-29T14:02:19.338 INFO:tasks.rados.rados.0.smithi017.stdout:11: writing smithi017107884-11 from 1552114 to 2168070 tid 2 2023-03-29T14:02:19.338 INFO:tasks.rados.rados.0.smithi017.stdout:12: write initial oid 12 2023-03-29T14:02:19.338 INFO:tasks.rados.rados.0.smithi017.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2023-03-29T14:02:19.344 INFO:tasks.rados.rados.0.smithi017.stdout:12: writing smithi017107884-12 from 584877 to 1128969 tid 1 2023-03-29T14:02:19.350 INFO:tasks.rados.rados.0.smithi017.stdout:12: writing smithi017107884-12 from 1610596 to 2172599 tid 2 2023-03-29T14:02:19.350 INFO:tasks.rados.rados.0.smithi017.stdout:13: write initial oid 13 2023-03-29T14:02:19.351 INFO:tasks.rados.rados.0.smithi017.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2023-03-29T14:02:19.357 INFO:tasks.rados.rados.0.smithi017.stdout:13: writing smithi017107884-13 from 601684 to 1221025 tid 1 2023-03-29T14:02:19.363 INFO:tasks.rados.rados.0.smithi017.stdout:13: writing smithi017107884-13 from 1669078 to 2201684 tid 2 2023-03-29T14:02:19.364 INFO:tasks.rados.rados.0.smithi017.stdout:14: write initial oid 14 2023-03-29T14:02:19.364 INFO:tasks.rados.rados.0.smithi017.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2023-03-29T14:02:19.371 INFO:tasks.rados.rados.0.smithi017.stdout:14: writing smithi017107884-14 from 618491 to 1313081 tid 1 2023-03-29T14:02:19.376 INFO:tasks.rados.rados.0.smithi017.stdout:14: writing smithi017107884-14 from 2011207 to 2218491 tid 2 2023-03-29T14:02:19.376 INFO:tasks.rados.rados.0.smithi017.stdout:15: write initial oid 15 2023-03-29T14:02:19.376 INFO:tasks.rados.rados.0.smithi017.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2023-03-29T14:02:19.384 INFO:tasks.rados.rados.0.smithi017.stdout:15: writing smithi017107884-15 from 635298 to 1405137 tid 1 2023-03-29T14:02:19.389 INFO:tasks.rados.rados.0.smithi017.stdout:15: writing smithi017107884-15 from 2069689 to 2235298 tid 2 2023-03-29T14:02:19.390 INFO:tasks.rados.rados.0.smithi017.stdout:16: write initial oid 16 2023-03-29T14:02:19.390 INFO:tasks.rados.rados.0.smithi017.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2023-03-29T14:02:19.395 INFO:tasks.rados.rados.0.smithi017.stdout:16: writing smithi017107884-16 from 652105 to 1097193 tid 1 2023-03-29T14:02:19.402 INFO:tasks.rados.rados.0.smithi017.stdout:16: writing smithi017107884-16 from 1728171 to 2181159 tid 2 2023-03-29T14:02:19.402 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.402 INFO:tasks.rados.rados.0.smithi017.stdout:4: finishing write tid 1 to smithi017107884-4 2023-03-29T14:02:19.402 INFO:tasks.rados.rados.0.smithi017.stdout:1: finishing write tid 1 to smithi017107884-1 2023-03-29T14:02:19.402 INFO:tasks.rados.rados.0.smithi017.stdout:2: finishing write tid 2 to smithi017107884-2 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:6: finishing write tid 1 to smithi017107884-6 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:7: finishing write tid 1 to smithi017107884-7 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:2: finishing write tid 3 to smithi017107884-2 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:2: finishing write tid 4 to smithi017107884-2 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:2: oid 2 updating version 0 to 3 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:2: oid 2 version 3 is already newer than 1 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:2: oid 2 version 3 is already newer than 3 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:2: done (15 left) 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:17: write initial oid 17 2023-03-29T14:02:19.403 INFO:tasks.rados.rados.0.smithi017.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2023-03-29T14:02:19.409 INFO:tasks.rados.rados.0.smithi017.stdout:17: writing smithi017107884-17 from 668912 to 1305602 tid 1 2023-03-29T14:02:19.415 INFO:tasks.rados.rados.0.smithi017.stdout:17: writing smithi017107884-17 from 1903006 to 2268912 tid 2 2023-03-29T14:02:19.415 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.415 INFO:tasks.rados.rados.0.smithi017.stdout:9: finishing write tid 1 to smithi017107884-9 2023-03-29T14:02:19.418 INFO:tasks.rados.rados.0.smithi017.stdout:1: finishing write tid 2 to smithi017107884-1 2023-03-29T14:02:19.423 INFO:tasks.rados.rados.0.smithi017.stdout:4: finishing write tid 2 to smithi017107884-4 2023-03-29T14:02:19.457 INFO:tasks.rados.rados.0.smithi017.stdout:16: finishing write tid 1 to smithi017107884-16 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:1: finishing write tid 3 to smithi017107884-1 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:1: finishing write tid 4 to smithi017107884-1 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:1: oid 1 updating version 0 to 3 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:1: oid 1 version 3 is already newer than 1 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:1: oid 1 version 3 is already newer than 3 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:1: done (15 left) 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:18: write initial oid 18 2023-03-29T14:02:19.459 INFO:tasks.rados.rados.0.smithi017.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2023-03-29T14:02:19.466 INFO:tasks.rados.rados.0.smithi017.stdout:18: writing smithi017107884-18 from 685719 to 1397658 tid 1 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:18: writing smithi017107884-18 from 1845135 to 2285719 tid 2 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:4: finishing write tid 3 to smithi017107884-4 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:6: finishing write tid 2 to smithi017107884-6 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:4: finishing write tid 4 to smithi017107884-4 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:4: oid 4 updating version 0 to 1 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:4: oid 4 updating version 1 to 3 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:4: oid 4 version 3 is already newer than 3 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:6: finishing write tid 3 to smithi017107884-6 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:6: finishing write tid 4 to smithi017107884-6 2023-03-29T14:02:19.472 INFO:tasks.rados.rados.0.smithi017.stdout:6: oid 6 updating version 0 to 3 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:6: oid 6 version 3 is already newer than 1 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:6: oid 6 version 3 is already newer than 3 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:5: finishing write tid 1 to smithi017107884-5 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:3: finishing write tid 1 to smithi017107884-3 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:4: done (15 left) 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:6: done (14 left) 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:19: write initial oid 19 2023-03-29T14:02:19.473 INFO:tasks.rados.rados.0.smithi017.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2023-03-29T14:02:19.483 INFO:tasks.rados.rados.0.smithi017.stdout:19: writing smithi017107884-19 from 702526 to 1489714 tid 1 2023-03-29T14:02:19.490 INFO:tasks.rados.rados.0.smithi017.stdout:19: writing smithi017107884-19 from 1903617 to 2302526 tid 2 2023-03-29T14:02:19.490 INFO:tasks.rados.rados.0.smithi017.stdout:20: write initial oid 20 2023-03-29T14:02:19.490 INFO:tasks.rados.rados.0.smithi017.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2023-03-29T14:02:19.498 INFO:tasks.rados.rados.0.smithi017.stdout:20: writing smithi017107884-20 from 719333 to 1181770 tid 1 2023-03-29T14:02:19.506 INFO:tasks.rados.rados.0.smithi017.stdout:20: writing smithi017107884-20 from 1962099 to 2319333 tid 2 2023-03-29T14:02:19.506 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.506 INFO:tasks.rados.rados.0.smithi017.stdout:14: finishing write tid 1 to smithi017107884-14 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: finishing write tid 2 to smithi017107884-7 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: finishing write tid 3 to smithi017107884-7 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: finishing write tid 4 to smithi017107884-7 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: oid 7 updating version 0 to 3 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: oid 7 version 3 is already newer than 1 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: oid 7 version 3 is already newer than 3 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:7: done (15 left) 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:21: write initial oid 21 2023-03-29T14:02:19.521 INFO:tasks.rados.rados.0.smithi017.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2023-03-29T14:02:19.530 INFO:tasks.rados.rados.0.smithi017.stdout:21: writing smithi017107884-21 from 736140 to 1273826 tid 1 2023-03-29T14:02:19.537 INFO:tasks.rados.rados.0.smithi017.stdout:21: writing smithi017107884-21 from 2020581 to 2336140 tid 2 2023-03-29T14:02:19.537 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.537 INFO:tasks.rados.rados.0.smithi017.stdout:16: finishing write tid 2 to smithi017107884-16 2023-03-29T14:02:19.544 INFO:tasks.rados.rados.0.smithi017.stdout:9: finishing write tid 2 to smithi017107884-9 2023-03-29T14:02:19.555 INFO:tasks.rados.rados.0.smithi017.stdout:8: finishing write tid 1 to smithi017107884-8 2023-03-29T14:02:19.564 INFO:tasks.rados.rados.0.smithi017.stdout:16: finishing write tid 3 to smithi017107884-16 2023-03-29T14:02:19.565 INFO:tasks.rados.rados.0.smithi017.stdout:16: finishing write tid 4 to smithi017107884-16 2023-03-29T14:02:19.565 INFO:tasks.rados.rados.0.smithi017.stdout:16: oid 16 updating version 0 to 1 2023-03-29T14:02:19.565 INFO:tasks.rados.rados.0.smithi017.stdout:16: oid 16 updating version 1 to 3 2023-03-29T14:02:19.565 INFO:tasks.rados.rados.0.smithi017.stdout:16: oid 16 version 3 is already newer than 3 2023-03-29T14:02:19.566 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2023-03-29T14:02:19.566 INFO:tasks.rados.rados.0.smithi017.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2023-03-29T14:02:19.566 INFO:tasks.rados.rados.0.smithi017.stdout:16: done (15 left) 2023-03-29T14:02:19.566 INFO:tasks.rados.rados.0.smithi017.stdout:22: write initial oid 22 2023-03-29T14:02:19.566 INFO:tasks.rados.rados.0.smithi017.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2023-03-29T14:02:19.576 INFO:tasks.rados.rados.0.smithi017.stdout:22: writing smithi017107884-22 from 752947 to 1365882 tid 1 2023-03-29T14:02:19.583 INFO:tasks.rados.rados.0.smithi017.stdout:22: writing smithi017107884-22 from 1962710 to 2352947 tid 2 2023-03-29T14:02:19.583 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.589 INFO:tasks.rados.rados.0.smithi017.stdout:9: finishing write tid 3 to smithi017107884-9 2023-03-29T14:02:19.594 INFO:tasks.rados.rados.0.smithi017.stdout:9: finishing write tid 4 to smithi017107884-9 2023-03-29T14:02:19.594 INFO:tasks.rados.rados.0.smithi017.stdout:9: oid 9 updating version 0 to 1 2023-03-29T14:02:19.594 INFO:tasks.rados.rados.0.smithi017.stdout:9: oid 9 updating version 1 to 3 2023-03-29T14:02:19.594 INFO:tasks.rados.rados.0.smithi017.stdout:9: oid 9 version 3 is already newer than 3 2023-03-29T14:02:19.594 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 3 (ObjNum 8 snap 0 seq_num 8) dirty exists 2023-03-29T14:02:19.594 INFO:tasks.rados.rados.0.smithi017.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2023-03-29T14:02:19.594 INFO:tasks.rados.rados.0.smithi017.stdout:9: done (15 left) 2023-03-29T14:02:19.595 INFO:tasks.rados.rados.0.smithi017.stdout:23: write initial oid 23 2023-03-29T14:02:19.595 INFO:tasks.rados.rados.0.smithi017.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2023-03-29T14:02:19.605 INFO:tasks.rados.rados.0.smithi017.stdout:23: writing smithi017107884-23 from 769754 to 1457938 tid 1 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:23: writing smithi017107884-23 from 2021192 to 2369754 tid 2 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:5: finishing write tid 2 to smithi017107884-5 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: finishing write tid 2 to smithi017107884-14 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: finishing write tid 3 to smithi017107884-14 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: finishing write tid 4 to smithi017107884-14 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: oid 14 updating version 0 to 3 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: oid 14 version 3 is already newer than 1 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: oid 14 version 3 is already newer than 3 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 3 (ObjNum 13 snap 0 seq_num 13) dirty exists 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:14: done (15 left) 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:24: write initial oid 24 2023-03-29T14:02:19.612 INFO:tasks.rados.rados.0.smithi017.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2023-03-29T14:02:19.620 INFO:tasks.rados.rados.0.smithi017.stdout:24: writing smithi017107884-24 from 786561 to 1266347 tid 1 2023-03-29T14:02:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:24: writing smithi017107884-24 from 1796027 to 2263691 tid 2 2023-03-29T14:02:19.628 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.631 INFO:tasks.rados.rados.0.smithi017.stdout:3: finishing write tid 2 to smithi017107884-3 2023-03-29T14:02:19.633 INFO:tasks.rados.rados.0.smithi017.stdout:10: finishing write tid 1 to smithi017107884-10 2023-03-29T14:02:19.634 INFO:tasks.rados.rados.0.smithi017.stdout:5: finishing write tid 3 to smithi017107884-5 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:5: finishing write tid 4 to smithi017107884-5 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:5: oid 5 updating version 0 to 1 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:5: oid 5 updating version 1 to 3 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:5: oid 5 version 3 is already newer than 3 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:5: done (15 left) 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:25: write initial oid 25 2023-03-29T14:02:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2023-03-29T14:02:19.642 INFO:tasks.rados.rados.0.smithi017.stdout:25: writing smithi017107884-25 from 403368 to 958403 tid 1 2023-03-29T14:02:19.653 INFO:tasks.rados.rados.0.smithi017.stdout:25: writing smithi017107884-25 from 1454509 to 2182184 tid 2 2023-03-29T14:02:19.653 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.653 INFO:tasks.rados.rados.0.smithi017.stdout:8: finishing write tid 2 to smithi017107884-8 2023-03-29T14:02:19.653 INFO:tasks.rados.rados.0.smithi017.stdout:3: finishing write tid 3 to smithi017107884-3 2023-03-29T14:02:19.653 INFO:tasks.rados.rados.0.smithi017.stdout:21: finishing write tid 1 to smithi017107884-21 2023-03-29T14:02:19.653 INFO:tasks.rados.rados.0.smithi017.stdout:20: finishing write tid 1 to smithi017107884-20 2023-03-29T14:02:19.665 INFO:tasks.rados.rados.0.smithi017.stdout:3: finishing write tid 4 to smithi017107884-3 2023-03-29T14:02:19.665 INFO:tasks.rados.rados.0.smithi017.stdout:3: oid 3 updating version 0 to 4 2023-03-29T14:02:19.666 INFO:tasks.rados.rados.0.smithi017.stdout:3: oid 3 updating version 4 to 6 2023-03-29T14:02:19.666 INFO:tasks.rados.rados.0.smithi017.stdout:3: oid 3 version 6 is already newer than 6 2023-03-29T14:02:19.666 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 6 (ObjNum 2 snap 0 seq_num 2) dirty exists 2023-03-29T14:02:19.666 INFO:tasks.rados.rados.0.smithi017.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2023-03-29T14:02:19.666 INFO:tasks.rados.rados.0.smithi017.stdout:3: done (15 left) 2023-03-29T14:02:19.666 INFO:tasks.rados.rados.0.smithi017.stdout:26: write initial oid 26 2023-03-29T14:02:19.666 INFO:tasks.rados.rados.0.smithi017.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2023-03-29T14:02:19.674 INFO:tasks.rados.rados.0.smithi017.stdout:26: writing smithi017107884-26 from 420175 to 1050459 tid 1 2023-03-29T14:02:19.683 INFO:tasks.rados.rados.0.smithi017.stdout:26: writing smithi017107884-26 from 1796638 to 2267971 tid 2 2023-03-29T14:02:19.683 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.687 INFO:tasks.rados.rados.0.smithi017.stdout:22: finishing write tid 1 to smithi017107884-22 2023-03-29T14:02:19.694 INFO:tasks.rados.rados.0.smithi017.stdout:18: finishing write tid 1 to smithi017107884-18 2023-03-29T14:02:19.695 INFO:tasks.rados.rados.0.smithi017.stdout:8: finishing write tid 3 to smithi017107884-8 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:8: finishing write tid 4 to smithi017107884-8 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:8: oid 8 updating version 0 to 4 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:8: oid 8 updating version 4 to 6 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:8: oid 8 version 6 is already newer than 6 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:8: done (15 left) 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:27: write initial oid 27 2023-03-29T14:02:19.710 INFO:tasks.rados.rados.0.smithi017.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2023-03-29T14:02:19.720 INFO:tasks.rados.rados.0.smithi017.stdout:27: writing smithi017107884-27 from 436982 to 1142515 tid 1 2023-03-29T14:02:19.729 INFO:tasks.rados.rados.0.smithi017.stdout:27: writing smithi017107884-27 from 1855120 to 2436982 tid 2 2023-03-29T14:02:19.729 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.729 INFO:tasks.rados.rados.0.smithi017.stdout:10: finishing write tid 2 to smithi017107884-10 2023-03-29T14:02:19.729 INFO:tasks.rados.rados.0.smithi017.stdout:21: finishing write tid 2 to smithi017107884-21 2023-03-29T14:02:19.733 INFO:tasks.rados.rados.0.smithi017.stdout:21: finishing write tid 3 to smithi017107884-21 2023-03-29T14:02:19.733 INFO:tasks.rados.rados.0.smithi017.stdout:12: finishing write tid 1 to smithi017107884-12 2023-03-29T14:02:19.733 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:19.770 INFO:tasks.rados.rados.0.smithi017.stdout:21: finishing write tid 4 to smithi017107884-21 2023-03-29T14:02:19.770 INFO:tasks.rados.rados.0.smithi017.stdout:21: oid 21 updating version 0 to 6 2023-03-29T14:02:19.770 INFO:tasks.rados.rados.0.smithi017.stdout:21: oid 21 version 6 is already newer than 4 2023-03-29T14:02:19.770 INFO:tasks.rados.rados.0.smithi017.stdout:21: oid 21 version 6 is already newer than 6 2023-03-29T14:02:19.770 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2023-03-29T14:02:19.770 INFO:tasks.rados.rados.0.smithi017.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:18: finishing write tid 2 to smithi017107884-18 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:10: finishing write tid 3 to smithi017107884-10 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:10: finishing write tid 4 to smithi017107884-10 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:10: oid 10 updating version 0 to 6 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:10: oid 10 version 6 is already newer than 4 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:10: oid 10 version 6 is already newer than 6 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 6 (ObjNum 9 snap 0 seq_num 9) dirty exists 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:18: finishing write tid 3 to smithi017107884-18 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:10: done (15 left) 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:21: done (14 left) 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:28: write initial oid 28 2023-03-29T14:02:19.771 INFO:tasks.rados.rados.0.smithi017.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2023-03-29T14:02:19.779 INFO:tasks.rados.rados.0.smithi017.stdout:28: writing smithi017107884-28 from 453789 to 1234571 tid 1 2023-03-29T14:02:19.788 INFO:tasks.rados.rados.0.smithi017.stdout:28: writing smithi017107884-28 from 1913602 to 2388604 tid 2 2023-03-29T14:02:19.788 INFO:tasks.rados.rados.0.smithi017.stdout:29: write initial oid 29 2023-03-29T14:02:19.788 INFO:tasks.rados.rados.0.smithi017.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2023-03-29T14:02:19.795 INFO:tasks.rados.rados.0.smithi017.stdout:29: writing smithi017107884-29 from 470596 to 926627 tid 1 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:29: writing smithi017107884-29 from 1572084 to 2190744 tid 2 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:18: finishing write tid 4 to smithi017107884-18 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:18: oid 18 updating version 0 to 6 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:18: oid 18 version 6 is already newer than 6 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:18: oid 18 version 6 is already newer than 4 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 6 (ObjNum 17 snap 0 seq_num 17) dirty exists 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:20: finishing write tid 2 to smithi017107884-20 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:25: finishing write tid 1 to smithi017107884-25 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:11: finishing write tid 1 to smithi017107884-11 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:20: finishing write tid 3 to smithi017107884-20 2023-03-29T14:02:19.806 INFO:tasks.rados.rados.0.smithi017.stdout:18: done (15 left) 2023-03-29T14:02:19.807 INFO:tasks.rados.rados.0.smithi017.stdout:30: write initial oid 30 2023-03-29T14:02:19.807 INFO:tasks.rados.rados.0.smithi017.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2023-03-29T14:02:19.813 INFO:tasks.rados.rados.0.smithi017.stdout:30: writing smithi017107884-30 from 487403 to 1018683 tid 1 2023-03-29T14:02:19.822 INFO:tasks.rados.rados.0.smithi017.stdout:30: writing smithi017107884-30 from 1514213 to 1992884 tid 2 2023-03-29T14:02:19.822 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:22: finishing write tid 2 to smithi017107884-22 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:20: finishing write tid 4 to smithi017107884-20 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:20: oid 20 updating version 0 to 1 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:20: oid 20 updating version 1 to 3 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:20: oid 20 version 3 is already newer than 3 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:20: done (15 left) 2023-03-29T14:02:19.823 INFO:tasks.rados.rados.0.smithi017.stdout:31: write initial oid 31 2023-03-29T14:02:19.824 INFO:tasks.rados.rados.0.smithi017.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2023-03-29T14:02:19.830 INFO:tasks.rados.rados.0.smithi017.stdout:31: writing smithi017107884-31 from 504210 to 1110739 tid 1 2023-03-29T14:02:19.839 INFO:tasks.rados.rados.0.smithi017.stdout:31: writing smithi017107884-31 from 1572695 to 2195024 tid 2 2023-03-29T14:02:19.839 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.839 INFO:tasks.rados.rados.0.smithi017.stdout:26: finishing write tid 1 to smithi017107884-26 2023-03-29T14:02:19.847 INFO:tasks.rados.rados.0.smithi017.stdout:22: finishing write tid 3 to smithi017107884-22 2023-03-29T14:02:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:19.876+0000 7f6a8e01e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:19.879 INFO:tasks.rados.rados.0.smithi017.stdout:17: finishing write tid 1 to smithi017107884-17 2023-03-29T14:02:19.881 INFO:tasks.rados.rados.0.smithi017.stdout:19: finishing write tid 1 to smithi017107884-19 2023-03-29T14:02:19.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:19.881+0000 7f6a8e01e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:19.889 INFO:tasks.rados.rados.0.smithi017.stdout:22: finishing write tid 4 to smithi017107884-22 2023-03-29T14:02:19.889 INFO:tasks.rados.rados.0.smithi017.stdout:22: oid 22 updating version 0 to 4 2023-03-29T14:02:19.889 INFO:tasks.rados.rados.0.smithi017.stdout:22: oid 22 updating version 4 to 6 2023-03-29T14:02:19.889 INFO:tasks.rados.rados.0.smithi017.stdout:22: oid 22 version 6 is already newer than 6 2023-03-29T14:02:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 6 (ObjNum 21 snap 0 seq_num 21) dirty exists 2023-03-29T14:02:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2023-03-29T14:02:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:22: done (15 left) 2023-03-29T14:02:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:32: write initial oid 32 2023-03-29T14:02:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2023-03-29T14:02:19.900 INFO:tasks.rados.rados.0.smithi017.stdout:32: writing smithi017107884-32 from 521017 to 1319148 tid 1 2023-03-29T14:02:19.907 INFO:tasks.rados.rados.0.smithi017.stdout:32: writing smithi017107884-32 from 1747530 to 2229870 tid 2 2023-03-29T14:02:19.908 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.908 INFO:tasks.rados.rados.0.smithi017.stdout:12: finishing write tid 2 to smithi017107884-12 2023-03-29T14:02:19.908 INFO:tasks.rados.rados.0.smithi017.stdout:12: finishing write tid 3 to smithi017107884-12 2023-03-29T14:02:19.918 INFO:tasks.rados.rados.0.smithi017.stdout:12: finishing write tid 4 to smithi017107884-12 2023-03-29T14:02:19.918 INFO:tasks.rados.rados.0.smithi017.stdout:12: oid 12 updating version 0 to 6 2023-03-29T14:02:19.918 INFO:tasks.rados.rados.0.smithi017.stdout:12: oid 12 version 6 is already newer than 4 2023-03-29T14:02:19.918 INFO:tasks.rados.rados.0.smithi017.stdout:12: oid 12 version 6 is already newer than 6 2023-03-29T14:02:19.918 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2023-03-29T14:02:19.919 INFO:tasks.rados.rados.0.smithi017.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2023-03-29T14:02:19.919 INFO:tasks.rados.rados.0.smithi017.stdout:12: done (15 left) 2023-03-29T14:02:19.919 INFO:tasks.rados.rados.0.smithi017.stdout:33: write initial oid 33 2023-03-29T14:02:19.919 INFO:tasks.rados.rados.0.smithi017.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2023-03-29T14:02:19.926 INFO:tasks.rados.rados.0.smithi017.stdout:33: writing smithi017107884-33 from 537824 to 1011204 tid 1 2023-03-29T14:02:19.938 INFO:tasks.rados.rados.0.smithi017.stdout:33: writing smithi017107884-33 from 1806012 to 2432010 tid 2 2023-03-29T14:02:19.938 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: finishing write tid 2 to smithi017107884-26 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: finishing write tid 3 to smithi017107884-26 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: finishing write tid 4 to smithi017107884-26 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: oid 26 updating version 0 to 7 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: oid 26 updating version 7 to 9 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: oid 26 version 9 is already newer than 9 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:31: finishing write tid 1 to smithi017107884-31 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:25: finishing write tid 2 to smithi017107884-25 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:30: finishing write tid 1 to smithi017107884-30 2023-03-29T14:02:19.939 INFO:tasks.rados.rados.0.smithi017.stdout:26: done (15 left) 2023-03-29T14:02:19.940 INFO:tasks.rados.rados.0.smithi017.stdout:34: write initial oid 34 2023-03-29T14:02:19.940 INFO:tasks.rados.rados.0.smithi017.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2023-03-29T14:02:19.945 INFO:tasks.rados.rados.0.smithi017.stdout:34: writing smithi017107884-34 from 554631 to 1103260 tid 1 2023-03-29T14:02:19.954 INFO:tasks.rados.rados.0.smithi017.stdout:34: writing smithi017107884-34 from 1748141 to 2234150 tid 2 2023-03-29T14:02:19.954 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:19.955 INFO:tasks.rados.rados.0.smithi017.stdout:24: finishing write tid 1 to smithi017107884-24 2023-03-29T14:02:19.980 INFO:tasks.rados.rados.0.smithi017.stdout:25: finishing write tid 3 to smithi017107884-25 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:25: finishing write tid 4 to smithi017107884-25 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:25: oid 25 updating version 0 to 6 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:25: oid 25 version 6 is already newer than 4 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:25: oid 25 version 6 is already newer than 6 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:25: done (15 left) 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:35: write initial oid 35 2023-03-29T14:02:19.982 INFO:tasks.rados.rados.0.smithi017.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2023-03-29T14:02:19.991 INFO:tasks.rados.rados.0.smithi017.stdout:35: writing smithi017107884-35 from 571438 to 1195316 tid 1 2023-03-29T14:02:20.001 INFO:tasks.rados.rados.0.smithi017.stdout:35: writing smithi017107884-35 from 1806623 to 2436290 tid 2 2023-03-29T14:02:20.001 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.002 INFO:tasks.rados.rados.0.smithi017.stdout:28: finishing write tid 1 to smithi017107884-28 2023-03-29T14:02:20.006 INFO:tasks.rados.rados.0.smithi017.stdout:11: finishing write tid 2 to smithi017107884-11 2023-03-29T14:02:20.010 INFO:tasks.rados.rados.0.smithi017.stdout:27: finishing write tid 1 to smithi017107884-27 2023-03-29T14:02:20.019 INFO:tasks.rados.rados.0.smithi017.stdout:11: finishing write tid 3 to smithi017107884-11 2023-03-29T14:02:20.022 INFO:tasks.rados.rados.0.smithi017.stdout:19: finishing write tid 2 to smithi017107884-19 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:11: finishing write tid 4 to smithi017107884-11 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:11: oid 11 updating version 0 to 9 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:11: oid 11 version 9 is already newer than 9 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:11: oid 11 version 9 is already newer than 7 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 11 v 9 (ObjNum 10 snap 0 seq_num 10) dirty exists 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:11: done (15 left) 2023-03-29T14:02:20.028 INFO:tasks.rados.rados.0.smithi017.stdout:36: write initial oid 36 2023-03-29T14:02:20.029 INFO:tasks.rados.rados.0.smithi017.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2023-03-29T14:02:20.038 INFO:tasks.rados.rados.0.smithi017.stdout:36: writing smithi017107884-36 from 588245 to 1287372 tid 1 2023-03-29T14:02:20.046 INFO:tasks.rados.rados.0.smithi017.stdout:36: writing smithi017107884-36 from 1865105 to 2354783 tid 2 2023-03-29T14:02:20.046 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.046 INFO:tasks.rados.rados.0.smithi017.stdout:31: finishing write tid 2 to smithi017107884-31 2023-03-29T14:02:20.047 INFO:tasks.rados.rados.0.smithi017.stdout:31: finishing write tid 3 to smithi017107884-31 2023-03-29T14:02:20.048 INFO:tasks.rados.rados.0.smithi017.stdout:31: finishing write tid 4 to smithi017107884-31 2023-03-29T14:02:20.048 INFO:tasks.rados.rados.0.smithi017.stdout:31: oid 31 updating version 0 to 7 2023-03-29T14:02:20.048 INFO:tasks.rados.rados.0.smithi017.stdout:31: oid 31 updating version 7 to 9 2023-03-29T14:02:20.049 INFO:tasks.rados.rados.0.smithi017.stdout:31: oid 31 version 9 is already newer than 9 2023-03-29T14:02:20.049 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 9 (ObjNum 30 snap 0 seq_num 30) dirty exists 2023-03-29T14:02:20.049 INFO:tasks.rados.rados.0.smithi017.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2023-03-29T14:02:20.049 INFO:tasks.rados.rados.0.smithi017.stdout:31: done (15 left) 2023-03-29T14:02:20.049 INFO:tasks.rados.rados.0.smithi017.stdout:37: write initial oid 37 2023-03-29T14:02:20.049 INFO:tasks.rados.rados.0.smithi017.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2023-03-29T14:02:20.059 INFO:tasks.rados.rados.0.smithi017.stdout:37: writing smithi017107884-37 from 605052 to 1379428 tid 1 2023-03-29T14:02:20.068 INFO:tasks.rados.rados.0.smithi017.stdout:37: writing smithi017107884-37 from 1923587 to 2556923 tid 2 2023-03-29T14:02:20.069 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.069 INFO:tasks.rados.rados.0.smithi017.stdout:13: finishing write tid 1 to smithi017107884-13 2023-03-29T14:02:20.070 INFO:tasks.rados.rados.0.smithi017.stdout:19: finishing write tid 3 to smithi017107884-19 2023-03-29T14:02:20.086 INFO:tasks.rados.rados.0.smithi017.stdout:19: finishing write tid 4 to smithi017107884-19 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:19: oid 19 updating version 0 to 9 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:19: oid 19 version 9 is already newer than 7 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:19: oid 19 version 9 is already newer than 9 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 9 (ObjNum 18 snap 0 seq_num 18) dirty exists 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:19: done (15 left) 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:38: write initial oid 38 2023-03-29T14:02:20.087 INFO:tasks.rados.rados.0.smithi017.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2023-03-29T14:02:20.094 INFO:tasks.rados.rados.0.smithi017.stdout:38: writing smithi017107884-38 from 621859 to 1071484 tid 1 2023-03-29T14:02:20.106 INFO:tasks.rados.rados.0.smithi017.stdout:38: writing smithi017107884-38 from 1865716 to 2621859 tid 2 2023-03-29T14:02:20.107 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.107 INFO:tasks.rados.rados.0.smithi017.stdout:17: finishing write tid 2 to smithi017107884-17 2023-03-29T14:02:20.127 INFO:tasks.rados.rados.0.smithi017.stdout:30: finishing write tid 2 to smithi017107884-30 2023-03-29T14:02:20.141 INFO:tasks.rados.rados.0.smithi017.stdout:24: finishing write tid 2 to smithi017107884-24 2023-03-29T14:02:20.149 INFO:tasks.rados.rados.0.smithi017.stdout:17: finishing write tid 3 to smithi017107884-17 2023-03-29T14:02:20.149 INFO:tasks.rados.rados.0.smithi017.stdout:24: finishing write tid 3 to smithi017107884-24 2023-03-29T14:02:20.150 INFO:tasks.rados.rados.0.smithi017.stdout:17: finishing write tid 4 to smithi017107884-17 2023-03-29T14:02:20.150 INFO:tasks.rados.rados.0.smithi017.stdout:17: oid 17 updating version 0 to 9 2023-03-29T14:02:20.150 INFO:tasks.rados.rados.0.smithi017.stdout:17: oid 17 version 9 is already newer than 9 2023-03-29T14:02:20.150 INFO:tasks.rados.rados.0.smithi017.stdout:17: oid 17 version 9 is already newer than 7 2023-03-29T14:02:20.151 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 9 (ObjNum 16 snap 0 seq_num 16) dirty exists 2023-03-29T14:02:20.151 INFO:tasks.rados.rados.0.smithi017.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2023-03-29T14:02:20.151 INFO:tasks.rados.rados.0.smithi017.stdout:17: done (15 left) 2023-03-29T14:02:20.151 INFO:tasks.rados.rados.0.smithi017.stdout:39: write initial oid 39 2023-03-29T14:02:20.151 INFO:tasks.rados.rados.0.smithi017.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2023-03-29T14:02:20.156 INFO:tasks.rados.rados.0.smithi017.stdout:39: writing smithi017107884-39 from 638666 to 1163540 tid 1 2023-03-29T14:02:20.165 INFO:tasks.rados.rados.0.smithi017.stdout:39: writing smithi017107884-39 from 1924198 to 2561203 tid 2 2023-03-29T14:02:20.165 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.165 INFO:tasks.rados.rados.0.smithi017.stdout:34: finishing write tid 1 to smithi017107884-34 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:28: finishing write tid 2 to smithi017107884-28 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:27: finishing write tid 2 to smithi017107884-27 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:24: finishing write tid 4 to smithi017107884-24 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:24: oid 24 updating version 0 to 3 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:24: oid 24 version 3 is already newer than 1 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:24: oid 24 version 3 is already newer than 3 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 3 (ObjNum 23 snap 0 seq_num 23) dirty exists 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:24: done (15 left) 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:40: write initial oid 40 2023-03-29T14:02:20.166 INFO:tasks.rados.rados.0.smithi017.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2023-03-29T14:02:20.173 INFO:tasks.rados.rados.0.smithi017.stdout:40: writing smithi017107884-40 from 655473 to 1371949 tid 1 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout:40: writing smithi017107884-40 from 2099033 to 2655473 tid 2 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout:30: finishing write tid 3 to smithi017107884-30 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout:30: finishing write tid 4 to smithi017107884-30 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout:30: oid 30 updating version 0 to 1 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout:30: oid 30 updating version 1 to 3 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout:30: oid 30 version 3 is already newer than 3 2023-03-29T14:02:20.182 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 3 (ObjNum 29 snap 0 seq_num 29) dirty exists 2023-03-29T14:02:20.183 INFO:tasks.rados.rados.0.smithi017.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2023-03-29T14:02:20.183 INFO:tasks.rados.rados.0.smithi017.stdout:15: finishing write tid 1 to smithi017107884-15 2023-03-29T14:02:20.183 INFO:tasks.rados.rados.0.smithi017.stdout:30: done (15 left) 2023-03-29T14:02:20.183 INFO:tasks.rados.rados.0.smithi017.stdout:41: write initial oid 41 2023-03-29T14:02:20.183 INFO:tasks.rados.rados.0.smithi017.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2023-03-29T14:02:20.189 INFO:tasks.rados.rados.0.smithi017.stdout:41: writing smithi017107884-41 from 672280 to 1464005 tid 1 2023-03-29T14:02:20.196 INFO:tasks.rados.rados.0.smithi017.stdout:41: writing smithi017107884-41 from 2157515 to 2672280 tid 2 2023-03-29T14:02:20.196 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.196 INFO:tasks.rados.rados.0.smithi017.stdout:28: finishing write tid 3 to smithi017107884-28 2023-03-29T14:02:20.215 INFO:tasks.rados.rados.0.smithi017.stdout:13: finishing write tid 2 to smithi017107884-13 2023-03-29T14:02:20.234 INFO:tasks.rados.rados.0.smithi017.stdout:33: finishing write tid 1 to smithi017107884-33 2023-03-29T14:02:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:27: finishing write tid 3 to smithi017107884-27 2023-03-29T14:02:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:28: finishing write tid 4 to smithi017107884-28 2023-03-29T14:02:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:28: oid 28 updating version 0 to 4 2023-03-29T14:02:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:28: oid 28 updating version 4 to 6 2023-03-29T14:02:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:28: oid 28 version 6 is already newer than 6 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:27: finishing write tid 4 to smithi017107884-27 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:27: oid 27 updating version 0 to 12 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:27: oid 27 version 12 is already newer than 10 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:27: oid 27 version 12 is already newer than 12 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 12 (ObjNum 26 snap 0 seq_num 26) dirty exists 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:27: done (15 left) 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:28: done (14 left) 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:42: write initial oid 42 2023-03-29T14:02:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2023-03-29T14:02:20.243 INFO:tasks.rados.rados.0.smithi017.stdout:42: writing smithi017107884-42 from 689087 to 1156061 tid 1 2023-03-29T14:02:20.251 INFO:tasks.rados.rados.0.smithi017.stdout:42: writing smithi017107884-42 from 1699644 to 2483976 tid 2 2023-03-29T14:02:20.251 INFO:tasks.rados.rados.0.smithi017.stdout:43: write initial oid 43 2023-03-29T14:02:20.251 INFO:tasks.rados.rados.0.smithi017.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2023-03-29T14:02:20.257 INFO:tasks.rados.rados.0.smithi017.stdout:43: writing smithi017107884-43 from 705894 to 1248117 tid 1 2023-03-29T14:02:20.264 INFO:tasks.rados.rados.0.smithi017.stdout:43: writing smithi017107884-43 from 1758126 to 2402469 tid 2 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:37: finishing write tid 1 to smithi017107884-37 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:13: finishing write tid 3 to smithi017107884-13 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:13: finishing write tid 4 to smithi017107884-13 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:13: oid 13 updating version 0 to 7 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:13: oid 13 updating version 7 to 9 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:13: oid 13 version 9 is already newer than 9 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 9 (ObjNum 12 snap 0 seq_num 12) dirty exists 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:35: finishing write tid 1 to smithi017107884-35 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:13: done (15 left) 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:44: write initial oid 44 2023-03-29T14:02:20.265 INFO:tasks.rados.rados.0.smithi017.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2023-03-29T14:02:20.272 INFO:tasks.rados.rados.0.smithi017.stdout:44: writing smithi017107884-44 from 722701 to 1340173 tid 1 2023-03-29T14:02:20.280 INFO:tasks.rados.rados.0.smithi017.stdout:44: writing smithi017107884-44 from 1816608 to 2604609 tid 2 2023-03-29T14:02:20.280 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.280 INFO:tasks.rados.rados.0.smithi017.stdout:36: finishing write tid 1 to smithi017107884-36 2023-03-29T14:02:20.280 INFO:tasks.rados.rados.0.smithi017.stdout:15: finishing write tid 2 to smithi017107884-15 2023-03-29T14:02:20.281 INFO:tasks.rados.rados.0.smithi017.stdout:15: finishing write tid 3 to smithi017107884-15 2023-03-29T14:02:20.291 INFO:tasks.rados.rados.0.smithi017.stdout:39: finishing write tid 1 to smithi017107884-39 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:15: finishing write tid 4 to smithi017107884-15 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:15: oid 15 updating version 0 to 12 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:15: oid 15 version 12 is already newer than 10 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:15: oid 15 version 12 is already newer than 12 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 12 (ObjNum 14 snap 0 seq_num 14) dirty exists 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:15: done (15 left) 2023-03-29T14:02:20.306 INFO:tasks.rados.rados.0.smithi017.stdout:45: write initial oid 45 2023-03-29T14:02:20.307 INFO:tasks.rados.rados.0.smithi017.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2023-03-29T14:02:20.314 INFO:tasks.rados.rados.0.smithi017.stdout:45: writing smithi017107884-45 from 739508 to 1432229 tid 1 2023-03-29T14:02:20.320 INFO:tasks.rados.rados.0.smithi017.stdout:45: writing smithi017107884-45 from 1875090 to 2523102 tid 2 2023-03-29T14:02:20.320 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.320 INFO:tasks.rados.rados.0.smithi017.stdout:29: finishing write tid 1 to smithi017107884-29 2023-03-29T14:02:20.327 INFO:tasks.rados.rados.0.smithi017.stdout:34: finishing write tid 2 to smithi017107884-34 2023-03-29T14:02:20.349 INFO:tasks.rados.rados.0.smithi017.stdout:37: finishing write tid 2 to smithi017107884-37 2023-03-29T14:02:20.362 INFO:tasks.rados.rados.0.smithi017.stdout:37: finishing write tid 3 to smithi017107884-37 2023-03-29T14:02:20.363 INFO:tasks.rados.rados.0.smithi017.stdout:37: finishing write tid 4 to smithi017107884-37 2023-03-29T14:02:20.363 INFO:tasks.rados.rados.0.smithi017.stdout:37: oid 37 updating version 0 to 6 2023-03-29T14:02:20.363 INFO:tasks.rados.rados.0.smithi017.stdout:37: oid 37 version 6 is already newer than 4 2023-03-29T14:02:20.363 INFO:tasks.rados.rados.0.smithi017.stdout:37: oid 37 version 6 is already newer than 6 2023-03-29T14:02:20.364 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2023-03-29T14:02:20.364 INFO:tasks.rados.rados.0.smithi017.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2023-03-29T14:02:20.364 INFO:tasks.rados.rados.0.smithi017.stdout:37: done (15 left) 2023-03-29T14:02:20.364 INFO:tasks.rados.rados.0.smithi017.stdout:46: write initial oid 46 2023-03-29T14:02:20.364 INFO:tasks.rados.rados.0.smithi017.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2023-03-29T14:02:20.371 INFO:tasks.rados.rados.0.smithi017.stdout:46: writing smithi017107884-46 from 756315 to 1524285 tid 1 2023-03-29T14:02:20.379 INFO:tasks.rados.rados.0.smithi017.stdout:46: writing smithi017107884-46 from 1933572 to 2725242 tid 2 2023-03-29T14:02:20.379 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.379 INFO:tasks.rados.rados.0.smithi017.stdout:34: finishing write tid 3 to smithi017107884-34 2023-03-29T14:02:20.379 INFO:tasks.rados.rados.0.smithi017.stdout:34: finishing write tid 4 to smithi017107884-34 2023-03-29T14:02:20.379 INFO:tasks.rados.rados.0.smithi017.stdout:34: oid 34 updating version 0 to 4 2023-03-29T14:02:20.379 INFO:tasks.rados.rados.0.smithi017.stdout:34: oid 34 updating version 4 to 6 2023-03-29T14:02:20.380 INFO:tasks.rados.rados.0.smithi017.stdout:34: oid 34 version 6 is already newer than 6 2023-03-29T14:02:20.380 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2023-03-29T14:02:20.380 INFO:tasks.rados.rados.0.smithi017.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2023-03-29T14:02:20.380 INFO:tasks.rados.rados.0.smithi017.stdout:38: finishing write tid 1 to smithi017107884-38 2023-03-29T14:02:20.380 INFO:tasks.rados.rados.0.smithi017.stdout:34: done (15 left) 2023-03-29T14:02:20.380 INFO:tasks.rados.rados.0.smithi017.stdout:47: write initial oid 47 2023-03-29T14:02:20.380 INFO:tasks.rados.rados.0.smithi017.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2023-03-29T14:02:20.386 INFO:tasks.rados.rados.0.smithi017.stdout:47: writing smithi017107884-47 from 773122 to 1332694 tid 1 2023-03-29T14:02:20.394 INFO:tasks.rados.rados.0.smithi017.stdout:47: writing smithi017107884-47 from 1992054 to 2643735 tid 2 2023-03-29T14:02:20.394 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.394 INFO:tasks.rados.rados.0.smithi017.stdout:40: finishing write tid 1 to smithi017107884-40 2023-03-29T14:02:20.394 INFO:tasks.rados.rados.0.smithi017.stdout:23: finishing write tid 1 to smithi017107884-23 2023-03-29T14:02:20.408 INFO:tasks.rados.rados.0.smithi017.stdout:35: finishing write tid 2 to smithi017107884-35 2023-03-29T14:02:20.421 INFO:tasks.rados.rados.0.smithi017.stdout:39: finishing write tid 2 to smithi017107884-39 2023-03-29T14:02:20.422 INFO:tasks.rados.rados.0.smithi017.stdout:39: finishing write tid 3 to smithi017107884-39 2023-03-29T14:02:20.423 INFO:tasks.rados.rados.0.smithi017.stdout:39: finishing write tid 4 to smithi017107884-39 2023-03-29T14:02:20.423 INFO:tasks.rados.rados.0.smithi017.stdout:39: oid 39 updating version 0 to 12 2023-03-29T14:02:20.423 INFO:tasks.rados.rados.0.smithi017.stdout:39: oid 39 version 12 is already newer than 12 2023-03-29T14:02:20.423 INFO:tasks.rados.rados.0.smithi017.stdout:39: oid 39 version 12 is already newer than 10 2023-03-29T14:02:20.423 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2023-03-29T14:02:20.423 INFO:tasks.rados.rados.0.smithi017.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2023-03-29T14:02:20.424 INFO:tasks.rados.rados.0.smithi017.stdout:39: done (15 left) 2023-03-29T14:02:20.424 INFO:tasks.rados.rados.0.smithi017.stdout:48: write initial oid 48 2023-03-29T14:02:20.424 INFO:tasks.rados.rados.0.smithi017.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2023-03-29T14:02:20.431 INFO:tasks.rados.rados.0.smithi017.stdout:48: writing smithi017107884-48 from 789929 to 1424750 tid 1 2023-03-29T14:02:20.439 INFO:tasks.rados.rados.0.smithi017.stdout:48: writing smithi017107884-48 from 2050536 to 2789929 tid 2 2023-03-29T14:02:20.439 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.439 INFO:tasks.rados.rados.0.smithi017.stdout:33: finishing write tid 2 to smithi017107884-33 2023-03-29T14:02:20.439 INFO:tasks.rados.rados.0.smithi017.stdout:33: finishing write tid 3 to smithi017107884-33 2023-03-29T14:02:20.439 INFO:tasks.rados.rados.0.smithi017.stdout:23: finishing write tid 2 to smithi017107884-23 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:33: finishing write tid 4 to smithi017107884-33 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:33: oid 33 updating version 0 to 10 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:33: oid 33 updating version 10 to 12 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:33: oid 33 version 12 is already newer than 12 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 12 (ObjNum 32 snap 0 seq_num 32) dirty exists 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:33: done (15 left) 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:49: write initial oid 49 2023-03-29T14:02:20.441 INFO:tasks.rados.rados.0.smithi017.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2023-03-29T14:02:20.447 INFO:tasks.rados.rados.0.smithi017.stdout:49: writing smithi017107884-49 from 406736 to 1116806 tid 1 2023-03-29T14:02:20.454 INFO:tasks.rados.rados.0.smithi017.stdout:49: writing smithi017107884-49 from 1709018 to 2364368 tid 2 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:23: finishing write tid 3 to smithi017107884-23 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:35: finishing write tid 3 to smithi017107884-35 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:35: finishing write tid 4 to smithi017107884-35 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:35: oid 35 updating version 0 to 1 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:35: oid 35 updating version 1 to 3 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:35: oid 35 version 3 is already newer than 3 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 3 (ObjNum 34 snap 0 seq_num 34) dirty exists 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:36: finishing write tid 2 to smithi017107884-36 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:35: done (15 left) 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:50: write initial oid 50 2023-03-29T14:02:20.455 INFO:tasks.rados.rados.0.smithi017.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2023-03-29T14:02:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:50: writing smithi017107884-50 from 423543 to 1208862 tid 1 2023-03-29T14:02:20.469 INFO:tasks.rados.rados.0.smithi017.stdout:50: writing smithi017107884-50 from 1767500 to 2566508 tid 2 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:36: finishing write tid 3 to smithi017107884-36 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:40: finishing write tid 2 to smithi017107884-40 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:36: finishing write tid 4 to smithi017107884-36 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:36: oid 36 updating version 0 to 3 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:36: oid 36 version 3 is already newer than 3 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:36: oid 36 version 3 is already newer than 1 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 3 (ObjNum 35 snap 0 seq_num 35) dirty exists 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:36: done (15 left) 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:51: write (excl) oid 2 current snap is 0 2023-03-29T14:02:20.470 INFO:tasks.rados.rados.0.smithi017.stdout:51: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2023-03-29T14:02:20.475 INFO:tasks.rados.rados.0.smithi017.stdout:51: writing smithi017107884-2 from 440350 to 900918 tid 1 2023-03-29T14:02:20.480 INFO:tasks.rados.rados.0.smithi017.stdout:51: writing smithi017107884-2 from 1309629 to 1852295 tid 2 2023-03-29T14:02:20.486 INFO:tasks.rados.rados.0.smithi017.stdout:51: writing smithi017107884-2 from 2323973 to 2840350 tid 3 2023-03-29T14:02:20.486 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.486 INFO:tasks.rados.rados.0.smithi017.stdout:41: finishing write tid 1 to smithi017107884-41 2023-03-29T14:02:20.486 INFO:tasks.rados.rados.0.smithi017.stdout:23: finishing write tid 4 to smithi017107884-23 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:23: oid 23 updating version 0 to 15 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:23: oid 23 version 15 is already newer than 13 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:23: oid 23 version 15 is already newer than 15 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 15 (ObjNum 22 snap 0 seq_num 22) dirty exists 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:23: done (15 left) 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:52: write (excl) oid 14 current snap is 0 2023-03-29T14:02:20.487 INFO:tasks.rados.rados.0.smithi017.stdout:52: seq_num 51 ranges {457157=535817,1768111=402677} 2023-03-29T14:02:20.491 INFO:tasks.rados.rados.0.smithi017.stdout:52: writing smithi017107884-14 from 457157 to 992974 tid 1 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:52: writing smithi017107884-14 from 1768111 to 2170788 tid 2 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:40: finishing write tid 3 to smithi017107884-40 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:40: finishing write tid 4 to smithi017107884-40 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:40: oid 40 updating version 0 to 9 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:40: oid 40 version 9 is already newer than 9 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:40: oid 40 version 9 is already newer than 7 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 9 (ObjNum 39 snap 0 seq_num 39) dirty exists 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2023-03-29T14:02:20.498 INFO:tasks.rados.rados.0.smithi017.stdout:40: done (15 left) 2023-03-29T14:02:20.499 INFO:tasks.rados.rados.0.smithi017.stdout:53: read oid 20 snap -1 2023-03-29T14:02:20.499 INFO:tasks.rados.rados.0.smithi017.stdout:53: expect (ObjNum 19 snap 0 seq_num 19) 2023-03-29T14:02:20.499 INFO:tasks.rados.rados.0.smithi017.stdout: waiting on 16 2023-03-29T14:02:20.506 INFO:tasks.rados.rados.0.smithi017.stdout:38: finishing write tid 2 to smithi017107884-38 2023-03-29T14:02:20.509 INFO:tasks.rados.rados.0.smithi017.stdout:32: finishing write tid 1 to smithi017107884-32 2023-03-29T14:02:20.509 INFO:tasks.rados.rados.0.smithi017.stdout:29: finishing write tid 2 to smithi017107884-29 2023-03-29T14:02:20.529 INFO:tasks.rados.rados.0.smithi017.stdout:48: finishing write tid 1 to smithi017107884-48 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:38: finishing write tid 3 to smithi017107884-38 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:38: finishing write tid 4 to smithi017107884-38 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:38: oid 38 updating version 0 to 3 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:38: oid 38 version 3 is already newer than 3 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:38: oid 38 version 3 is already newer than 1 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 3 (ObjNum 37 snap 0 seq_num 37) dirty exists 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:38: done (15 left) 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:54: delete oid 23 current snap is 0 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:29: finishing write tid 3 to smithi017107884-29 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:29: finishing write tid 4 to smithi017107884-29 2023-03-29T14:02:20.547 INFO:tasks.rados.rados.0.smithi017.stdout:29: oid 29 updating version 0 to 6 2023-03-29T14:02:20.548 INFO:tasks.rados.rados.0.smithi017.stdout:29: oid 29 version 6 is already newer than 4 2023-03-29T14:02:20.548 INFO:tasks.rados.rados.0.smithi017.stdout:29: oid 29 version 6 is already newer than 6 2023-03-29T14:02:20.548 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2023-03-29T14:02:20.548 INFO:tasks.rados.rados.0.smithi017.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2023-03-29T14:02:20.548 INFO:tasks.rados.rados.0.smithi017.stdout:32: finishing write tid 2 to smithi017107884-32 2023-03-29T14:02:20.569 INFO:tasks.rados.rados.0.smithi017.stdout:42: finishing write tid 1 to smithi017107884-42 2023-03-29T14:02:20.570 INFO:tasks.rados.rados.0.smithi017.stdout:52: finishing write tid 1 to smithi017107884-14 2023-03-29T14:02:20.583 INFO:tasks.rados.rados.0.smithi017.stdout:44: finishing write tid 1 to smithi017107884-44 2023-03-29T14:02:20.587 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:20.587 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:20.587 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:20.588 INFO:tasks.rados.rados.0.smithi017.stdout:32: finishing write tid 3 to smithi017107884-32 2023-03-29T14:02:20.595 INFO:tasks.rados.rados.0.smithi017.stdout:48: finishing write tid 2 to smithi017107884-48 2023-03-29T14:02:20.595 INFO:tasks.rados.rados.0.smithi017.stdout:32: finishing write tid 4 to smithi017107884-32 2023-03-29T14:02:20.595 INFO:tasks.rados.rados.0.smithi017.stdout:32: oid 32 updating version 0 to 18 2023-03-29T14:02:20.595 INFO:tasks.rados.rados.0.smithi017.stdout:32: oid 32 version 18 is already newer than 18 2023-03-29T14:02:20.595 INFO:tasks.rados.rados.0.smithi017.stdout:32: oid 32 version 18 is already newer than 16 2023-03-29T14:02:20.596 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 18 (ObjNum 31 snap 0 seq_num 31) dirty exists 2023-03-29T14:02:20.596 INFO:tasks.rados.rados.0.smithi017.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2023-03-29T14:02:20.603 INFO:tasks.rados.rados.0.smithi017.stdout:48: finishing write tid 3 to smithi017107884-48 2023-03-29T14:02:20.610 INFO:tasks.rados.rados.0.smithi017.stdout:45: finishing write tid 1 to smithi017107884-45 2023-03-29T14:02:20.614 INFO:tasks.rados.rados.0.smithi017.stdout:48: finishing write tid 4 to smithi017107884-48 2023-03-29T14:02:20.614 INFO:tasks.rados.rados.0.smithi017.stdout:48: oid 48 updating version 0 to 15 2023-03-29T14:02:20.614 INFO:tasks.rados.rados.0.smithi017.stdout:48: oid 48 version 15 is already newer than 13 2023-03-29T14:02:20.614 INFO:tasks.rados.rados.0.smithi017.stdout:48: oid 48 version 15 is already newer than 15 2023-03-29T14:02:20.614 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 15 (ObjNum 47 snap 0 seq_num 47) dirty exists 2023-03-29T14:02:20.614 INFO:tasks.rados.rados.0.smithi017.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2023-03-29T14:02:20.635 INFO:tasks.rados.rados.0.smithi017.stdout:43: finishing write tid 1 to smithi017107884-43 2023-03-29T14:02:20.635 INFO:tasks.rados.rados.0.smithi017.stdout:42: finishing write tid 2 to smithi017107884-42 2023-03-29T14:02:20.652 INFO:tasks.rados.rados.0.smithi017.stdout:49: finishing write tid 1 to smithi017107884-49 2023-03-29T14:02:20.656 INFO:tasks.rados.rados.0.smithi017.stdout:42: finishing write tid 3 to smithi017107884-42 2023-03-29T14:02:20.660 INFO:tasks.rados.rados.0.smithi017.stdout:41: finishing write tid 2 to smithi017107884-41 2023-03-29T14:02:20.665 INFO:tasks.rados.rados.0.smithi017.stdout:42: finishing write tid 4 to smithi017107884-42 2023-03-29T14:02:20.665 INFO:tasks.rados.rados.0.smithi017.stdout:42: oid 42 updating version 0 to 10 2023-03-29T14:02:20.665 INFO:tasks.rados.rados.0.smithi017.stdout:42: oid 42 updating version 10 to 12 2023-03-29T14:02:20.666 INFO:tasks.rados.rados.0.smithi017.stdout:42: oid 42 version 12 is already newer than 12 2023-03-29T14:02:20.666 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2023-03-29T14:02:20.666 INFO:tasks.rados.rados.0.smithi017.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2023-03-29T14:02:20.669 INFO:tasks.rados.rados.0.smithi017.stdout:41: finishing write tid 3 to smithi017107884-41 2023-03-29T14:02:20.679 INFO:tasks.rados.rados.0.smithi017.stdout:41: finishing write tid 4 to smithi017107884-41 2023-03-29T14:02:20.679 INFO:tasks.rados.rados.0.smithi017.stdout:41: oid 41 updating version 0 to 12 2023-03-29T14:02:20.679 INFO:tasks.rados.rados.0.smithi017.stdout:41: oid 41 version 12 is already newer than 12 2023-03-29T14:02:20.679 INFO:tasks.rados.rados.0.smithi017.stdout:41: oid 41 version 12 is already newer than 10 2023-03-29T14:02:20.679 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 12 (ObjNum 40 snap 0 seq_num 40) dirty exists 2023-03-29T14:02:20.679 INFO:tasks.rados.rados.0.smithi017.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2023-03-29T14:02:20.681 INFO:tasks.rados.rados.0.smithi017.stdout:52: finishing write tid 2 to smithi017107884-14 2023-03-29T14:02:20.685 INFO:tasks.rados.rados.0.smithi017.stdout:52: finishing write tid 3 to smithi017107884-14 2023-03-29T14:02:20.687 INFO:tasks.rados.rados.0.smithi017.stdout:52: finishing write tid 4 to smithi017107884-14 2023-03-29T14:02:20.687 INFO:tasks.rados.rados.0.smithi017.stdout:52: oid 14 updating version 0 to 10 2023-03-29T14:02:20.687 INFO:tasks.rados.rados.0.smithi017.stdout:52: oid 14 updating version 10 to 12 2023-03-29T14:02:20.687 INFO:tasks.rados.rados.0.smithi017.stdout:52: oid 14 version 12 is already newer than 12 2023-03-29T14:02:20.687 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 12 (ObjNum 51 snap 0 seq_num 51) dirty exists 2023-03-29T14:02:20.687 INFO:tasks.rados.rados.0.smithi017.stdout:52: left oid 14 (ObjNum 51 snap 0 seq_num 51) 2023-03-29T14:02:20.688 INFO:tasks.rados.rados.0.smithi017.stdout:44: finishing write tid 2 to smithi017107884-44 2023-03-29T14:02:20.690 INFO:tasks.rados.rados.0.smithi017.stdout:44: finishing write tid 3 to smithi017107884-44 2023-03-29T14:02:20.696 INFO:tasks.rados.rados.0.smithi017.stdout:44: finishing write tid 4 to smithi017107884-44 2023-03-29T14:02:20.696 INFO:tasks.rados.rados.0.smithi017.stdout:44: oid 44 updating version 0 to 6 2023-03-29T14:02:20.696 INFO:tasks.rados.rados.0.smithi017.stdout:44: oid 44 version 6 is already newer than 4 2023-03-29T14:02:20.696 INFO:tasks.rados.rados.0.smithi017.stdout:44: oid 44 version 6 is already newer than 6 2023-03-29T14:02:20.696 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2023-03-29T14:02:20.696 INFO:tasks.rados.rados.0.smithi017.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2023-03-29T14:02:20.697 INFO:tasks.rados.rados.0.smithi017.stdout:45: finishing write tid 2 to smithi017107884-45 2023-03-29T14:02:20.700 INFO:tasks.rados.rados.0.smithi017.stdout:45: finishing write tid 3 to smithi017107884-45 2023-03-29T14:02:20.707 INFO:tasks.rados.rados.0.smithi017.stdout:45: finishing write tid 4 to smithi017107884-45 2023-03-29T14:02:20.707 INFO:tasks.rados.rados.0.smithi017.stdout:45: oid 45 updating version 0 to 9 2023-03-29T14:02:20.708 INFO:tasks.rados.rados.0.smithi017.stdout:45: oid 45 version 9 is already newer than 9 2023-03-29T14:02:20.708 INFO:tasks.rados.rados.0.smithi017.stdout:45: oid 45 version 9 is already newer than 7 2023-03-29T14:02:20.708 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 9 (ObjNum 44 snap 0 seq_num 44) dirty exists 2023-03-29T14:02:20.708 INFO:tasks.rados.rados.0.smithi017.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2023-03-29T14:02:20.714 INFO:tasks.rados.rados.0.smithi017.stdout:43: finishing write tid 2 to smithi017107884-43 2023-03-29T14:02:20.716 INFO:tasks.rados.rados.0.smithi017.stdout:49: finishing write tid 2 to smithi017107884-49 2023-03-29T14:02:20.718 INFO:tasks.rados.rados.0.smithi017.stdout:43: finishing write tid 3 to smithi017107884-43 2023-03-29T14:02:20.718 INFO:tasks.rados.rados.0.smithi017.stdout:49: finishing write tid 3 to smithi017107884-49 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:49: finishing write tid 4 to smithi017107884-49 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:49: oid 49 updating version 0 to 21 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:49: oid 49 version 21 is already newer than 21 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:49: oid 49 version 21 is already newer than 19 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 21 (ObjNum 48 snap 0 seq_num 48) dirty exists 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:43: finishing write tid 4 to smithi017107884-43 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:43: oid 43 updating version 0 to 6 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:43: oid 43 version 6 is already newer than 6 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:43: oid 43 version 6 is already newer than 4 2023-03-29T14:02:20.726 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 6 (ObjNum 42 snap 0 seq_num 42) dirty exists 2023-03-29T14:02:20.727 INFO:tasks.rados.rados.0.smithi017.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2023-03-29T14:02:20.727 INFO:tasks.rados.rados.0.smithi017.stdout:47: finishing write tid 1 to smithi017107884-47 2023-03-29T14:02:20.740 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:20.734+0000 7fd9ab489700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:20.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:20.743+0000 7fd9ab489700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:20.750 INFO:tasks.rados.rados.0.smithi017.stdout:46: finishing write tid 1 to smithi017107884-46 2023-03-29T14:02:20.769 INFO:tasks.rados.rados.0.smithi017.stdout:50: finishing write tid 1 to smithi017107884-50 2023-03-29T14:02:20.783 INFO:tasks.rados.rados.0.smithi017.stdout:51: finishing write tid 1 to smithi017107884-2 2023-03-29T14:02:20.785 INFO:tasks.rados.rados.0.smithi017.stdout:47: finishing write tid 2 to smithi017107884-47 2023-03-29T14:02:20.788 INFO:tasks.rados.rados.0.smithi017.stdout:47: finishing write tid 3 to smithi017107884-47 2023-03-29T14:02:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:47: finishing write tid 4 to smithi017107884-47 2023-03-29T14:02:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:47: oid 47 updating version 0 to 15 2023-03-29T14:02:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:47: oid 47 version 15 is already newer than 15 2023-03-29T14:02:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:47: oid 47 version 15 is already newer than 13 2023-03-29T14:02:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 15 (ObjNum 46 snap 0 seq_num 46) dirty exists 2023-03-29T14:02:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2023-03-29T14:02:20.799 INFO:tasks.rados.rados.0.smithi017.stdout:46: finishing write tid 2 to smithi017107884-46 2023-03-29T14:02:20.808 INFO:tasks.rados.rados.0.smithi017.stdout:46: finishing write tid 3 to smithi017107884-46 2023-03-29T14:02:20.810 INFO:tasks.rados.rados.0.smithi017.stdout:46: finishing write tid 4 to smithi017107884-46 2023-03-29T14:02:20.810 INFO:tasks.rados.rados.0.smithi017.stdout:46: oid 46 updating version 0 to 10 2023-03-29T14:02:20.810 INFO:tasks.rados.rados.0.smithi017.stdout:46: oid 46 updating version 10 to 12 2023-03-29T14:02:20.810 INFO:tasks.rados.rados.0.smithi017.stdout:46: oid 46 version 12 is already newer than 12 2023-03-29T14:02:20.810 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 12 (ObjNum 45 snap 0 seq_num 45) dirty exists 2023-03-29T14:02:20.810 INFO:tasks.rados.rados.0.smithi017.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2023-03-29T14:02:20.822 INFO:tasks.rados.rados.0.smithi017.stdout:51: finishing write tid 2 to smithi017107884-2 2023-03-29T14:02:20.824 INFO:tasks.rados.rados.0.smithi017.stdout:53: expect (ObjNum 19 snap 0 seq_num 19) 2023-03-29T14:02:20.927 INFO:tasks.rados.rados.0.smithi017.stdout:51: finishing write tid 3 to smithi017107884-2 2023-03-29T14:02:20.927 INFO:tasks.rados.rados.0.smithi017.stdout:51: finishing write tid 4 to smithi017107884-2 2023-03-29T14:02:20.927 INFO:tasks.rados.rados.0.smithi017.stdout:51: finishing write tid 5 to smithi017107884-2 2023-03-29T14:02:20.927 INFO:tasks.rados.rados.0.smithi017.stdout:51: oid 2 updating version 0 to 25 2023-03-29T14:02:20.927 INFO:tasks.rados.rados.0.smithi017.stdout:51: oid 2 version 25 is already newer than 22 2023-03-29T14:02:20.927 INFO:tasks.rados.rados.0.smithi017.stdout:51: oid 2 version 25 is already newer than 25 2023-03-29T14:02:20.927 INFO:tasks.rados.rados.0.smithi017.stdout:51: oid 2 version 25 is already newer than 25 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 25 (ObjNum 50 snap 0 seq_num 50) dirty exists 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:51: left oid 2 (ObjNum 50 snap 0 seq_num 50) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:50: finishing write tid 2 to smithi017107884-50 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:50: finishing write tid 3 to smithi017107884-50 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:29: done (15 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:32: done (14 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:41: done (13 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:42: done (12 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:43: done (11 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:44: done (10 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:45: done (9 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:46: done (8 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:47: done (7 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:48: done (6 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:49: done (5 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:51: done (4 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:52: done (3 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:53: done (2 left) 2023-03-29T14:02:20.928 INFO:tasks.rados.rados.0.smithi017.stdout:54: done (1 left) 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:55: write oid 28 current snap is 0 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:50: finishing write tid 4 to smithi017107884-50 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:50: oid 50 updating version 0 to 9 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:50: oid 50 version 9 is already newer than 7 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:50: oid 50 version 9 is already newer than 9 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2023-03-29T14:02:20.929 INFO:tasks.rados.rados.0.smithi017.stdout:55: seq_num 53 ranges {490771=686315,1885075=406346} 2023-03-29T14:02:20.933 INFO:tasks.rados.rados.0.smithi017.stdout:55: writing smithi017107884-28 from 490771 to 1177086 tid 1 2023-03-29T14:02:20.940 INFO:tasks.rados.rados.0.smithi017.stdout:55: writing smithi017107884-28 from 1885075 to 2291421 tid 2 2023-03-29T14:02:20.940 INFO:tasks.rados.rados.0.smithi017.stdout:50: done (1 left) 2023-03-29T14:02:20.940 INFO:tasks.rados.rados.0.smithi017.stdout:56: write oid 25 current snap is 0 2023-03-29T14:02:20.940 INFO:tasks.rados.rados.0.smithi017.stdout:56: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2023-03-29T14:02:20.945 INFO:tasks.rados.rados.0.smithi017.stdout:56: writing smithi017107884-25 from 507578 to 985495 tid 1 2023-03-29T14:02:20.951 INFO:tasks.rados.rados.0.smithi017.stdout:56: writing smithi017107884-25 from 1543557 to 2093561 tid 2 2023-03-29T14:02:20.956 INFO:tasks.rados.rados.0.smithi017.stdout:56: writing smithi017107884-25 from 2833665 to 2907578 tid 3 2023-03-29T14:02:20.956 INFO:tasks.rados.rados.0.smithi017.stdout:57: write (excl) oid 13 current snap is 0 2023-03-29T14:02:20.956 INFO:tasks.rados.rados.0.smithi017.stdout:57: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2023-03-29T14:02:20.962 INFO:tasks.rados.rados.0.smithi017.stdout:57: writing smithi017107884-13 from 524385 to 1077551 tid 1 2023-03-29T14:02:20.967 INFO:tasks.rados.rados.0.smithi017.stdout:57: writing smithi017107884-13 from 1602039 to 2012054 tid 2 2023-03-29T14:02:20.972 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:20.973 INFO:tasks.rados.rados.0.smithi017.stdout:57: writing smithi017107884-13 from 2577441 to 2924385 tid 3 2023-03-29T14:02:20.973 INFO:tasks.rados.rados.0.smithi017.stdout:58: read oid 43 snap -1 2023-03-29T14:02:20.973 INFO:tasks.rados.rados.0.smithi017.stdout:58: expect (ObjNum 42 snap 0 seq_num 42) 2023-03-29T14:02:20.973 INFO:tasks.rados.rados.0.smithi017.stdout:59: write (excl) oid 4 current snap is 0 2023-03-29T14:02:20.974 INFO:tasks.rados.rados.0.smithi017.stdout:59: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2023-03-29T14:02:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:20.979 INFO:tasks.rados.rados.0.smithi017.stdout:59: writing smithi017107884-4 from 541192 to 1169607 tid 1 2023-03-29T14:02:20.985 INFO:tasks.rados.rados.0.smithi017.stdout:59: writing smithi017107884-4 from 1660521 to 2214194 tid 2 2023-03-29T14:02:20.988 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-03-29T14:02:20.096342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":63,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":21016,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":20480,"kb_avail":377466344,"statfs":{"total":386547056640,"available":386525536256,"internally_reserved":0,"allocated":417792,"data_stored":1853720,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":20971520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262176,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":108,"num_write_kb":2092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000737"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":3,"reported_epoch":15,"state":"creating+activating","last_fresh":"2023-03-29T14:02:17.208576+0000","last_change":"2023-03-29T14:02:17.210106+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:17.208576+0000","last_undegraded":"2023-03-29T14:02:17.208576+0000","last_fullsized":"2023-03-29T14:02:17.208576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":["2","0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-03-29T14:02:16.192094+0000","last_change":"2023-03-29T14:02:16.192094+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.192094+0000","last_undegraded":"2023-03-29T14:02:16.192094+0000","last_fullsized":"2023-03-29T14:02:16.192094+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-03-29T14:02:16.192094+0000","last_change":"2023-03-29T14:02:16.192094+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.192094+0000","last_undegraded":"2023-03-29T14:02:16.192094+0000","last_fullsized":"2023-03-29T14:02:16.192094+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.272522+0000","last_change":"2023-03-29T14:02:16.272522+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.272522+0000","last_undegraded":"2023-03-29T14:02:16.272522+0000","last_fullsized":"2023-03-29T14:02:16.272522+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.241661+0000","last_change":"2023-03-29T14:02:16.241661+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.241661+0000","last_undegraded":"2023-03-29T14:02:16.241661+0000","last_fullsized":"2023-03-29T14:02:16.241661+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-03-29T14:02:16.192094+0000","last_change":"2023-03-29T14:02:16.192094+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.192094+0000","last_undegraded":"2023-03-29T14:02:16.192094+0000","last_fullsized":"2023-03-29T14:02:16.192094+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.227930+0000","last_change":"2023-03-29T14:02:16.227930+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.227930+0000","last_undegraded":"2023-03-29T14:02:16.227930+0000","last_fullsized":"2023-03-29T14:02:16.227930+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.258212+0000","last_change":"2023-03-29T14:02:16.258212+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.258212+0000","last_undegraded":"2023-03-29T14:02:16.258212+0000","last_fullsized":"2023-03-29T14:02:16.258212+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.201386+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:11.201386+0000","last_peered":"2023-03-29T14:02:11.201386+0000","last_clean":"2023-03-29T14:02:11.201386+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:11.201386+0000","last_undegraded":"2023-03-29T14:02:11.201386+0000","last_fullsized":"2023-03-29T14:02:11.201386+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":3,"reported_epoch":15,"state":"creating+activating","last_fresh":"2023-03-29T14:02:17.222110+0000","last_change":"2023-03-29T14:02:17.224008+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:17.222110+0000","last_undegraded":"2023-03-29T14:02:17.222110+0000","last_fullsized":"2023-03-29T14:02:17.222110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":["2","0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.203057+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:11.203057+0000","last_peered":"2023-03-29T14:02:11.203057+0000","last_clean":"2023-03-29T14:02:11.203057+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:11.203057+0000","last_undegraded":"2023-03-29T14:02:11.203057+0000","last_fullsized":"2023-03-29T14:02:11.203057+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.242606+0000","last_change":"2023-03-29T14:02:16.242606+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.242606+0000","last_undegraded":"2023-03-29T14:02:16.242606+0000","last_fullsized":"2023-03-29T14:02:16.242606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-03-29T14:02:16.218408+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:16.218408+0000","last_peered":"2023-03-29T14:02:16.218408+0000","last_clean":"2023-03-29T14:02:16.218408+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:16.218408+0000","last_undegraded":"2023-03-29T14:02:16.218408+0000","last_fullsized":"2023-03-29T14:02:16.218408+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.276237+0000","last_change":"2023-03-29T14:02:16.276237+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.276237+0000","last_undegraded":"2023-03-29T14:02:16.276237+0000","last_fullsized":"2023-03-29T14:02:16.276237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.200784+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:11.200784+0000","last_peered":"2023-03-29T14:02:11.200784+0000","last_clean":"2023-03-29T14:02:11.200784+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:11.200784+0000","last_undegraded":"2023-03-29T14:02:11.200784+0000","last_fullsized":"2023-03-29T14:02:11.200784+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":236,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:12.961665+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:12.961665+0000","last_peered":"2023-03-29T14:02:12.961665+0000","last_clean":"2023-03-29T14:02:12.961665+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:12.961665+0000","last_undegraded":"2023-03-29T14:02:12.961665+0000","last_fullsized":"2023-03-29T14:02:12.961665+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.241104+0000","last_change":"2023-03-29T14:02:16.241104+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.241104+0000","last_undegraded":"2023-03-29T14:02:16.241104+0000","last_fullsized":"2023-03-29T14:02:16.241104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.202470+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:11.202470+0000","last_peered":"2023-03-29T14:02:11.202470+0000","last_clean":"2023-03-29T14:02:11.202470+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:11.202470+0000","last_undegraded":"2023-03-29T14:02:11.202470+0000","last_fullsized":"2023-03-29T14:02:11.202470+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.258892+0000","last_change":"2023-03-29T14:02:16.258892+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.258892+0000","last_undegraded":"2023-03-29T14:02:16.258892+0000","last_fullsized":"2023-03-29T14:02:16.258892+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.204160+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:11.204160+0000","last_peered":"2023-03-29T14:02:11.204160+0000","last_clean":"2023-03-29T14:02:11.204160+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:11.204160+0000","last_undegraded":"2023-03-29T14:02:11.204160+0000","last_fullsized":"2023-03-29T14:02:11.204160+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-03-29T14:02:16.192094+0000","last_change":"2023-03-29T14:02:16.192094+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.192094+0000","last_undegraded":"2023-03-29T14:02:16.192094+0000","last_fullsized":"2023-03-29T14:02:16.192094+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.256578+0000","last_change":"2023-03-29T14:02:16.256578+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.256578+0000","last_undegraded":"2023-03-29T14:02:16.256578+0000","last_fullsized":"2023-03-29T14:02:16.256578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.209891+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:11.209891+0000","last_peered":"2023-03-29T14:02:11.209891+0000","last_clean":"2023-03-29T14:02:11.209891+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:11.209891+0000","last_undegraded":"2023-03-29T14:02:11.209891+0000","last_fullsized":"2023-03-29T14:02:11.209891+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-03-29T14:02:11.203681+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:11.203681+0000","last_peered":"2023-03-29T14:02:11.203681+0000","last_clean":"2023-03-29T14:02:11.203681+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:11.203681+0000","last_undegraded":"2023-03-29T14:02:11.203681+0000","last_fullsized":"2023-03-29T14:02:11.203681+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":14,"state":"creating+peering","last_fresh":"2023-03-29T14:02:16.230901+0000","last_change":"2023-03-29T14:02:16.230901+0000","last_active":"2023-03-29T14:02:16.192094+0000","last_peered":"2023-03-29T14:02:16.192094+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-03-29T14:02:16.230901+0000","last_undegraded":"2023-03-29T14:02:16.230901+0000","last_fullsized":"2023-03-29T14:02:16.230901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4864,"kb_used_data":56,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366976,"statfs":{"total":96636764160,"available":96631783424,"internally_reserved":0,"allocated":57344,"data_stored":19909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5408,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366432,"statfs":{"total":96636764160,"available":96631226368,"internally_reserved":0,"allocated":122880,"data_stored":613257,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5404,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366436,"statfs":{"total":96636764160,"available":96631230464,"internally_reserved":0,"allocated":118784,"data_stored":610277,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5340,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366500,"statfs":{"total":96636764160,"available":96631296000,"internally_reserved":0,"allocated":118784,"data_stored":610277,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:59: writing smithi017107884-4 from 2888511 to 2941192 tid 3 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:60: delete oid 48 current snap is 0 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:55: finishing write tid 1 to smithi017107884-28 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:55: finishing write tid 2 to smithi017107884-28 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:55: finishing write tid 3 to smithi017107884-28 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:55: finishing write tid 4 to smithi017107884-28 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:55: oid 28 updating version 0 to 9 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:55: oid 28 version 9 is already newer than 9 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:55: oid 28 version 9 is already newer than 9 2023-03-29T14:02:20.993 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 9 (ObjNum 53 snap 0 seq_num 53) dirty exists 2023-03-29T14:02:20.994 INFO:tasks.rados.rados.0.smithi017.stdout:55: left oid 28 (ObjNum 53 snap 0 seq_num 53) 2023-03-29T14:02:21.031 INFO:tasks.rados.rados.0.smithi017.stdout:56: finishing write tid 1 to smithi017107884-25 2023-03-29T14:02:21.032 INFO:tasks.rados.rados.0.smithi017.stdout:56: finishing write tid 2 to smithi017107884-25 2023-03-29T14:02:21.032 INFO:tasks.rados.rados.0.smithi017.stdout:56: finishing write tid 3 to smithi017107884-25 2023-03-29T14:02:21.052 INFO:tasks.rados.rados.0.smithi017.stdout:57: finishing write tid 1 to smithi017107884-13 2023-03-29T14:02:21.056 INFO:tasks.rados.rados.0.smithi017.stdout:56: finishing write tid 4 to smithi017107884-25 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:56: finishing write tid 5 to smithi017107884-25 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:56: oid 25 updating version 0 to 16 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:56: oid 25 version 16 is already newer than 16 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:56: oid 25 version 16 is already newer than 16 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:56: oid 25 version 16 is already newer than 16 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 16 (ObjNum 54 snap 0 seq_num 54) dirty exists 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:56: left oid 25 (ObjNum 54 snap 0 seq_num 54) 2023-03-29T14:02:21.057 INFO:tasks.rados.rados.0.smithi017.stdout:59: finishing write tid 1 to smithi017107884-4 2023-03-29T14:02:21.097 INFO:tasks.rados.rados.0.smithi017.stdout:58: expect (ObjNum 42 snap 0 seq_num 42) 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: finishing write tid 2 to smithi017107884-13 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: finishing write tid 3 to smithi017107884-13 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: finishing write tid 4 to smithi017107884-13 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: finishing write tid 5 to smithi017107884-13 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: oid 13 updating version 0 to 19 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: oid 13 version 19 is already newer than 19 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: oid 13 version 19 is already newer than 19 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: oid 13 version 19 is already newer than 16 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 19 (ObjNum 55 snap 0 seq_num 55) dirty exists 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: left oid 13 (ObjNum 55 snap 0 seq_num 55) 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:59: finishing write tid 2 to smithi017107884-4 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:55: done (5 left) 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:56: done (4 left) 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:57: done (3 left) 2023-03-29T14:02:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:58: done (2 left) 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:60: done (1 left) 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:61: delete oid 1 current snap is 0 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: finishing write tid 3 to smithi017107884-4 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: finishing write tid 4 to smithi017107884-4 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: finishing write tid 5 to smithi017107884-4 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: oid 4 updating version 0 to 19 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: oid 4 version 19 is already newer than 19 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: oid 4 version 19 is already newer than 16 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: oid 4 version 19 is already newer than 19 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 19 (ObjNum 56 snap 0 seq_num 56) dirty exists 2023-03-29T14:02:21.205 INFO:tasks.rados.rados.0.smithi017.stdout:59: left oid 4 (ObjNum 56 snap 0 seq_num 56) 2023-03-29T14:02:21.247 INFO:tasks.rados.rados.0.smithi017.stdout:59: done (1 left) 2023-03-29T14:02:21.247 INFO:tasks.rados.rados.0.smithi017.stdout:61: done (0 left) 2023-03-29T14:02:21.248 INFO:tasks.rados.rados.0.smithi017.stdout:62: delete oid 27 current snap is 0 2023-03-29T14:02:21.292 INFO:tasks.rados.rados.0.smithi017.stdout:62: done (0 left) 2023-03-29T14:02:21.293 INFO:tasks.rados.rados.0.smithi017.stdout:63: read oid 15 snap -1 2023-03-29T14:02:21.293 INFO:tasks.rados.rados.0.smithi017.stdout:63: expect (ObjNum 14 snap 0 seq_num 14) 2023-03-29T14:02:21.293 INFO:tasks.rados.rados.0.smithi017.stdout:64: snap_create 2023-03-29T14:02:21.308 INFO:tasks.rados.rados.0.smithi017.stdout:63: expect (ObjNum 14 snap 0 seq_num 14) 2023-03-29T14:02:22.215 INFO:tasks.rados.rados.0.smithi017.stdout:63: done (1 left) 2023-03-29T14:02:22.215 INFO:tasks.rados.rados.0.smithi017.stdout:64: done (0 left) 2023-03-29T14:02:22.216 INFO:tasks.rados.rados.0.smithi017.stdout:65: write oid 40 current snap is 1 2023-03-29T14:02:22.216 INFO:tasks.rados.rados.0.smithi017.stdout:65: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2023-03-29T14:02:22.222 INFO:tasks.rados.rados.0.smithi017.stdout:65: writing smithi017107884-40 from 608420 to 1137831 tid 1 2023-03-29T14:02:22.228 INFO:tasks.rados.rados.0.smithi017.stdout:65: writing smithi017107884-40 from 1778096 to 2339107 tid 2 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:65: writing smithi017107884-40 from 2881850 to 3008420 tid 3 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:66: read oid 48 snap 0 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:66: expect deleted 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:67: read oid 28 snap 0 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:67: expect (ObjNum 53 snap 0 seq_num 53) 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:68: read oid 13 snap -1 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:68: expect (ObjNum 55 snap 0 seq_num 55) 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:69: read oid 43 snap -1 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:69: expect (ObjNum 42 snap 0 seq_num 42) 2023-03-29T14:02:22.233 INFO:tasks.rados.rados.0.smithi017.stdout:70: read oid 36 snap -1 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:70: expect (ObjNum 35 snap 0 seq_num 35) 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:71: read oid 50 snap -1 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:71: expect (ObjNum 49 snap 0 seq_num 49) 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:72: read oid 8 snap 0 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:72: expect (ObjNum 7 snap 0 seq_num 7) 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:73: read oid 27 snap 0 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:73: expect deleted 2023-03-29T14:02:22.234 INFO:tasks.rados.rados.0.smithi017.stdout:74: snap_create 2023-03-29T14:02:22.292 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:22.298 INFO:tasks.rados.rados.0.smithi017.stdout:67: expect (ObjNum 53 snap 0 seq_num 53) 2023-03-29T14:02:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:22.431+0000 7f9a1e51c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:22.435+0000 7f9a1e51c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:22.454 INFO:tasks.rados.rados.0.smithi017.stdout:68: expect (ObjNum 55 snap 0 seq_num 55) 2023-03-29T14:02:22.602 INFO:tasks.rados.rados.0.smithi017.stdout:65: finishing write tid 1 to smithi017107884-40 2023-03-29T14:02:22.602 INFO:tasks.rados.rados.0.smithi017.stdout:65: finishing write tid 2 to smithi017107884-40 2023-03-29T14:02:22.602 INFO:tasks.rados.rados.0.smithi017.stdout:65: finishing write tid 3 to smithi017107884-40 2023-03-29T14:02:22.603 INFO:tasks.rados.rados.0.smithi017.stdout:72: expect (ObjNum 7 snap 0 seq_num 7) 2023-03-29T14:02:22.685 INFO:tasks.rados.rados.0.smithi017.stdout:65: finishing write tid 4 to smithi017107884-40 2023-03-29T14:02:22.685 INFO:tasks.rados.rados.0.smithi017.stdout:65: finishing write tid 5 to smithi017107884-40 2023-03-29T14:02:22.686 INFO:tasks.rados.rados.0.smithi017.stdout:65: oid 40 updating version 0 to 21 2023-03-29T14:02:22.686 INFO:tasks.rados.rados.0.smithi017.stdout:65: oid 40 version 21 is already newer than 21 2023-03-29T14:02:22.686 INFO:tasks.rados.rados.0.smithi017.stdout:65: oid 40 version 21 is already newer than 21 2023-03-29T14:02:22.686 INFO:tasks.rados.rados.0.smithi017.stdout:65: oid 40 version 21 is already newer than 21 2023-03-29T14:02:22.686 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 21 (ObjNum 60 snap 1 seq_num 60) dirty exists 2023-03-29T14:02:22.686 INFO:tasks.rados.rados.0.smithi017.stdout:65: left oid 40 (ObjNum 60 snap 1 seq_num 60) 2023-03-29T14:02:22.686 INFO:tasks.rados.rados.0.smithi017.stdout:69: expect (ObjNum 42 snap 0 seq_num 42) 2023-03-29T14:02:22.794 INFO:tasks.rados.rados.0.smithi017.stdout:70: expect (ObjNum 35 snap 0 seq_num 35) 2023-03-29T14:02:22.894 INFO:tasks.rados.rados.0.smithi017.stdout:71: expect (ObjNum 49 snap 0 seq_num 49) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:65: done (9 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:66: done (8 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:67: done (7 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:68: done (6 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:69: done (5 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:70: done (4 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:71: done (3 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:72: done (2 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:73: done (1 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:74: done (0 left) 2023-03-29T14:02:23.218 INFO:tasks.rados.rados.0.smithi017.stdout:75: write oid 45 current snap is 2 2023-03-29T14:02:23.219 INFO:tasks.rados.rados.0.smithi017.stdout:75: seq_num 61 ranges {625227=721013,1952931=704669} 2023-03-29T14:02:23.225 INFO:tasks.rados.rados.0.smithi017.stdout:75: writing smithi017107884-45 from 625227 to 1346240 tid 1 2023-03-29T14:02:23.233 INFO:tasks.rados.rados.0.smithi017.stdout:75: writing smithi017107884-45 from 1952931 to 2657600 tid 2 2023-03-29T14:02:23.233 INFO:tasks.rados.rados.0.smithi017.stdout:76: read oid 16 snap -1 2023-03-29T14:02:23.233 INFO:tasks.rados.rados.0.smithi017.stdout:76: expect (ObjNum 15 snap 0 seq_num 15) 2023-03-29T14:02:23.233 INFO:tasks.rados.rados.0.smithi017.stdout:77: write oid 46 current snap is 2 2023-03-29T14:02:23.233 INFO:tasks.rados.rados.0.smithi017.stdout:77: seq_num 62 ranges {642034=796262,1895060=564680} 2023-03-29T14:02:23.241 INFO:tasks.rados.rados.0.smithi017.stdout:77: writing smithi017107884-46 from 642034 to 1438296 tid 1 2023-03-29T14:02:23.247 INFO:tasks.rados.rados.0.smithi017.stdout:77: writing smithi017107884-46 from 1895060 to 2459740 tid 2 2023-03-29T14:02:23.247 INFO:tasks.rados.rados.0.smithi017.stdout:78: read oid 26 snap 0 2023-03-29T14:02:23.247 INFO:tasks.rados.rados.0.smithi017.stdout:78: expect (ObjNum 25 snap 0 seq_num 25) 2023-03-29T14:02:23.247 INFO:tasks.rados.rados.0.smithi017.stdout:79: read oid 7 snap -1 2023-03-29T14:02:23.247 INFO:tasks.rados.rados.0.smithi017.stdout:79: expect (ObjNum 6 snap 0 seq_num 6) 2023-03-29T14:02:23.247 INFO:tasks.rados.rados.0.smithi017.stdout:80: write oid 44 current snap is 2 2023-03-29T14:02:23.247 INFO:tasks.rados.rados.0.smithi017.stdout:80: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2023-03-29T14:02:23.252 INFO:tasks.rados.rados.0.smithi017.stdout:80: writing smithi017107884-44 from 658841 to 1130352 tid 1 2023-03-29T14:02:23.259 INFO:tasks.rados.rados.0.smithi017.stdout:80: writing smithi017107884-44 from 1553542 to 2261880 tid 2 2023-03-29T14:02:23.263 INFO:tasks.rados.rados.0.smithi017.stdout:80: writing smithi017107884-44 from 2964119 to 3058841 tid 3 2023-03-29T14:02:23.263 INFO:tasks.rados.rados.0.smithi017.stdout:81: read oid 30 snap -1 2023-03-29T14:02:23.263 INFO:tasks.rados.rados.0.smithi017.stdout:81: expect (ObjNum 29 snap 0 seq_num 29) 2023-03-29T14:02:23.264 INFO:tasks.rados.rados.0.smithi017.stdout:82: snap_remove snap 1 2023-03-29T14:02:24.222 INFO:tasks.rados.rados.0.smithi017.stdout:82: done (7 left) 2023-03-29T14:02:24.222 INFO:tasks.rados.rados.0.smithi017.stdout:83: delete oid 49 current snap is 2 2023-03-29T14:02:24.222 INFO:tasks.rados.rados.0.smithi017.stdout:76: expect (ObjNum 15 snap 0 seq_num 15) 2023-03-29T14:02:24.340 INFO:tasks.rados.rados.0.smithi017.stdout:81: expect (ObjNum 29 snap 0 seq_num 29) 2023-03-29T14:02:24.439 INFO:tasks.rados.rados.0.smithi017.stdout:79: expect (ObjNum 6 snap 0 seq_num 6) 2023-03-29T14:02:24.504 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:24.522 INFO:tasks.rados.rados.0.smithi017.stdout:78: expect (ObjNum 25 snap 0 seq_num 25) 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:75: finishing write tid 1 to smithi017107884-45 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:75: finishing write tid 2 to smithi017107884-45 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:75: finishing write tid 3 to smithi017107884-45 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:75: finishing write tid 4 to smithi017107884-45 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:75: oid 45 updating version 0 to 16 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:75: oid 45 version 16 is already newer than 16 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:75: oid 45 version 16 is already newer than 16 2023-03-29T14:02:24.617 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 16 (ObjNum 61 snap 2 seq_num 61) dirty exists 2023-03-29T14:02:24.618 INFO:tasks.rados.rados.0.smithi017.stdout:75: left oid 45 (ObjNum 61 snap 2 seq_num 61) 2023-03-29T14:02:24.618 INFO:tasks.rados.rados.0.smithi017.stdout:80: finishing write tid 1 to smithi017107884-44 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: finishing write tid 2 to smithi017107884-44 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: finishing write tid 3 to smithi017107884-44 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: finishing write tid 4 to smithi017107884-44 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: finishing write tid 5 to smithi017107884-44 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: oid 44 updating version 0 to 11 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: oid 44 version 11 is already newer than 11 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: oid 44 version 11 is already newer than 11 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: oid 44 version 11 is already newer than 11 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 11 (ObjNum 63 snap 2 seq_num 63) dirty exists 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:80: left oid 44 (ObjNum 63 snap 2 seq_num 63) 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:77: finishing write tid 1 to smithi017107884-46 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:77: finishing write tid 2 to smithi017107884-46 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:77: finishing write tid 3 to smithi017107884-46 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:77: finishing write tid 4 to smithi017107884-46 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:77: oid 46 updating version 0 to 20 2023-03-29T14:02:24.619 INFO:tasks.rados.rados.0.smithi017.stdout:77: oid 46 version 20 is already newer than 20 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:77: oid 46 version 20 is already newer than 20 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 20 (ObjNum 62 snap 2 seq_num 62) dirty exists 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:77: left oid 46 (ObjNum 62 snap 2 seq_num 62) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:75: done (7 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:76: done (6 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:77: done (5 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:78: done (4 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:79: done (3 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:80: done (2 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:81: done (1 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:83: done (0 left) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:84: read oid 23 snap -1 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:84: expect deleted 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:85: read oid 26 snap 0 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:85: expect (ObjNum 25 snap 0 seq_num 25) 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:86: write oid 2 current snap is 2 2023-03-29T14:02:24.620 INFO:tasks.rados.rados.0.smithi017.stdout:86: seq_num 65 ranges {692455=622009,2070506=712007} 2023-03-29T14:02:24.624 INFO:tasks.rados.rados.0.smithi017.stdout:86: writing smithi017107884-2 from 692455 to 1314464 tid 1 2023-03-29T14:02:24.634 INFO:tasks.rados.rados.0.smithi017.stdout:86: writing smithi017107884-2 from 2070506 to 2782513 tid 2 2023-03-29T14:02:24.634 INFO:tasks.rados.rados.0.smithi017.stdout:87: snap_create 2023-03-29T14:02:24.634 INFO:tasks.rados.rados.0.smithi017.stdout:85: expect (ObjNum 25 snap 0 seq_num 25) 2023-03-29T14:02:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:24.633+0000 7f066464f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:24.638+0000 7f066464f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:24.729 INFO:tasks.rados.rados.0.smithi017.stdout:86: finishing write tid 1 to smithi017107884-2 2023-03-29T14:02:24.745 INFO:tasks.rados.rados.0.smithi017.stdout:86: finishing write tid 2 to smithi017107884-2 2023-03-29T14:02:24.746 INFO:tasks.rados.rados.0.smithi017.stdout:86: finishing write tid 3 to smithi017107884-2 2023-03-29T14:02:24.746 INFO:tasks.rados.rados.0.smithi017.stdout:86: finishing write tid 4 to smithi017107884-2 2023-03-29T14:02:24.746 INFO:tasks.rados.rados.0.smithi017.stdout:86: oid 2 updating version 0 to 32 2023-03-29T14:02:24.746 INFO:tasks.rados.rados.0.smithi017.stdout:86: oid 2 version 32 is already newer than 32 2023-03-29T14:02:24.746 INFO:tasks.rados.rados.0.smithi017.stdout:86: oid 2 version 32 is already newer than 32 2023-03-29T14:02:24.746 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 32 (ObjNum 65 snap 2 seq_num 65) dirty exists 2023-03-29T14:02:24.746 INFO:tasks.rados.rados.0.smithi017.stdout:86: left oid 2 (ObjNum 65 snap 2 seq_num 65) 2023-03-29T14:02:25.225 INFO:tasks.rados.rados.0.smithi017.stdout:84: done (3 left) 2023-03-29T14:02:25.225 INFO:tasks.rados.rados.0.smithi017.stdout:85: done (2 left) 2023-03-29T14:02:25.225 INFO:tasks.rados.rados.0.smithi017.stdout:86: done (1 left) 2023-03-29T14:02:25.225 INFO:tasks.rados.rados.0.smithi017.stdout:87: done (0 left) 2023-03-29T14:02:25.225 INFO:tasks.rados.rados.0.smithi017.stdout:88: write oid 6 current snap is 3 2023-03-29T14:02:25.225 INFO:tasks.rados.rados.0.smithi017.stdout:88: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2023-03-29T14:02:25.232 INFO:tasks.rados.rados.0.smithi017.stdout:88: writing smithi017107884-6 from 709262 to 1406520 tid 1 2023-03-29T14:02:25.239 INFO:tasks.rados.rados.0.smithi017.stdout:88: writing smithi017107884-6 from 2012635 to 2584653 tid 2 2023-03-29T14:02:25.242 INFO:tasks.rados.rados.0.smithi017.stdout:88: writing smithi017107884-6 from 3046388 to 3109262 tid 3 2023-03-29T14:02:25.242 INFO:tasks.rados.rados.0.smithi017.stdout:89: write oid 7 current snap is 3 2023-03-29T14:02:25.242 INFO:tasks.rados.rados.0.smithi017.stdout:89: seq_num 67 ranges {726069=772507,2071117=715676} 2023-03-29T14:02:25.250 INFO:tasks.rados.rados.0.smithi017.stdout:89: writing smithi017107884-7 from 726069 to 1498576 tid 1 2023-03-29T14:02:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:89: writing smithi017107884-7 from 2071117 to 2786793 tid 2 2023-03-29T14:02:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:90: write oid 22 current snap is 3 2023-03-29T14:02:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:90: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2023-03-29T14:02:25.264 INFO:tasks.rados.rados.0.smithi017.stdout:90: writing smithi017107884-22 from 742876 to 1190632 tid 1 2023-03-29T14:02:25.271 INFO:tasks.rados.rados.0.smithi017.stdout:90: writing smithi017107884-22 from 1729599 to 2305286 tid 2 2023-03-29T14:02:25.275 INFO:tasks.rados.rados.0.smithi017.stdout:90: writing smithi017107884-22 from 3101234 to 3142876 tid 3 2023-03-29T14:02:25.275 INFO:tasks.rados.rados.0.smithi017.stdout:91: delete oid 14 current snap is 3 2023-03-29T14:02:25.387 INFO:tasks.rados.rados.0.smithi017.stdout:90: finishing write tid 1 to smithi017107884-22 2023-03-29T14:02:25.391 INFO:tasks.rados.rados.0.smithi017.stdout:88: finishing write tid 1 to smithi017107884-6 2023-03-29T14:02:25.396 INFO:tasks.rados.rados.0.smithi017.stdout:90: finishing write tid 2 to smithi017107884-22 2023-03-29T14:02:25.428 INFO:tasks.rados.rados.0.smithi017.stdout:90: finishing write tid 3 to smithi017107884-22 2023-03-29T14:02:25.432 INFO:tasks.rados.rados.0.smithi017.stdout:90: finishing write tid 4 to smithi017107884-22 2023-03-29T14:02:25.436 INFO:tasks.rados.rados.0.smithi017.stdout:90: finishing write tid 5 to smithi017107884-22 2023-03-29T14:02:25.436 INFO:tasks.rados.rados.0.smithi017.stdout:90: oid 22 updating version 0 to 11 2023-03-29T14:02:25.436 INFO:tasks.rados.rados.0.smithi017.stdout:90: oid 22 version 11 is already newer than 11 2023-03-29T14:02:25.436 INFO:tasks.rados.rados.0.smithi017.stdout:90: oid 22 version 11 is already newer than 11 2023-03-29T14:02:25.437 INFO:tasks.rados.rados.0.smithi017.stdout:90: oid 22 version 11 is already newer than 11 2023-03-29T14:02:25.437 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 11 (ObjNum 68 snap 3 seq_num 68) dirty exists 2023-03-29T14:02:25.437 INFO:tasks.rados.rados.0.smithi017.stdout:90: left oid 22 (ObjNum 68 snap 3 seq_num 68) 2023-03-29T14:02:25.441 INFO:tasks.rados.rados.0.smithi017.stdout:88: finishing write tid 2 to smithi017107884-6 2023-03-29T14:02:25.446 INFO:tasks.rados.rados.0.smithi017.stdout:88: finishing write tid 3 to smithi017107884-6 2023-03-29T14:02:25.447 INFO:tasks.rados.rados.0.smithi017.stdout:88: finishing write tid 4 to smithi017107884-6 2023-03-29T14:02:25.449 INFO:tasks.rados.rados.0.smithi017.stdout:88: finishing write tid 5 to smithi017107884-6 2023-03-29T14:02:25.449 INFO:tasks.rados.rados.0.smithi017.stdout:88: oid 6 updating version 0 to 18 2023-03-29T14:02:25.449 INFO:tasks.rados.rados.0.smithi017.stdout:88: oid 6 version 18 is already newer than 18 2023-03-29T14:02:25.449 INFO:tasks.rados.rados.0.smithi017.stdout:88: oid 6 version 18 is already newer than 18 2023-03-29T14:02:25.449 INFO:tasks.rados.rados.0.smithi017.stdout:88: oid 6 version 18 is already newer than 18 2023-03-29T14:02:25.449 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 18 (ObjNum 66 snap 3 seq_num 66) dirty exists 2023-03-29T14:02:25.449 INFO:tasks.rados.rados.0.smithi017.stdout:88: left oid 6 (ObjNum 66 snap 3 seq_num 66) 2023-03-29T14:02:25.466 INFO:tasks.rados.rados.0.smithi017.stdout:89: finishing write tid 1 to smithi017107884-7 2023-03-29T14:02:25.494 INFO:tasks.rados.rados.0.smithi017.stdout:89: finishing write tid 2 to smithi017107884-7 2023-03-29T14:02:25.496 INFO:tasks.rados.rados.0.smithi017.stdout:89: finishing write tid 3 to smithi017107884-7 2023-03-29T14:02:25.497 INFO:tasks.rados.rados.0.smithi017.stdout:89: finishing write tid 4 to smithi017107884-7 2023-03-29T14:02:25.497 INFO:tasks.rados.rados.0.smithi017.stdout:89: oid 7 updating version 0 to 13 2023-03-29T14:02:25.497 INFO:tasks.rados.rados.0.smithi017.stdout:89: oid 7 version 13 is already newer than 13 2023-03-29T14:02:25.497 INFO:tasks.rados.rados.0.smithi017.stdout:89: oid 7 version 13 is already newer than 13 2023-03-29T14:02:25.497 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 13 (ObjNum 67 snap 3 seq_num 67) dirty exists 2023-03-29T14:02:25.497 INFO:tasks.rados.rados.0.smithi017.stdout:89: left oid 7 (ObjNum 67 snap 3 seq_num 67) 2023-03-29T14:02:25.502 INFO:tasks.rados.rados.0.smithi017.stdout:88: done (3 left) 2023-03-29T14:02:25.502 INFO:tasks.rados.rados.0.smithi017.stdout:89: done (2 left) 2023-03-29T14:02:25.502 INFO:tasks.rados.rados.0.smithi017.stdout:90: done (1 left) 2023-03-29T14:02:25.502 INFO:tasks.rados.rados.0.smithi017.stdout:91: done (0 left) 2023-03-29T14:02:25.502 INFO:tasks.rados.rados.0.smithi017.stdout:92: write (excl) oid 44 current snap is 3 2023-03-29T14:02:25.502 INFO:tasks.rados.rados.0.smithi017.stdout:92: seq_num 70 ranges {776490=714607,2246563=579356} 2023-03-29T14:02:25.510 INFO:tasks.rados.rados.0.smithi017.stdout:92: writing smithi017107884-44 from 776490 to 1491097 tid 1 2023-03-29T14:02:25.517 INFO:tasks.rados.rados.0.smithi017.stdout:92: writing smithi017107884-44 from 2246563 to 2825919 tid 2 2023-03-29T14:02:25.517 INFO:tasks.rados.rados.0.smithi017.stdout:93: write oid 22 current snap is 3 2023-03-29T14:02:25.518 INFO:tasks.rados.rados.0.smithi017.stdout:93: seq_num 71 ranges {793297=789856,2305045=723014} 2023-03-29T14:02:25.526 INFO:tasks.rados.rados.0.smithi017.stdout:93: writing smithi017107884-22 from 793297 to 1583153 tid 1 2023-03-29T14:02:25.534 INFO:tasks.rados.rados.0.smithi017.stdout:93: writing smithi017107884-22 from 2305045 to 3028059 tid 2 2023-03-29T14:02:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:94: read oid 23 snap -1 2023-03-29T14:02:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:94: expect deleted 2023-03-29T14:02:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:95: read oid 29 snap -1 2023-03-29T14:02:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:95: expect (ObjNum 28 snap 0 seq_num 28) 2023-03-29T14:02:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:96: delete oid 10 current snap is 3 2023-03-29T14:02:25.563 INFO:tasks.rados.rados.0.smithi017.stdout:92: finishing write tid 1 to smithi017107884-44 2023-03-29T14:02:25.611 INFO:tasks.rados.rados.0.smithi017.stdout:93: finishing write tid 1 to smithi017107884-22 2023-03-29T14:02:25.611 INFO:tasks.rados.rados.0.smithi017.stdout:93: finishing write tid 2 to smithi017107884-22 2023-03-29T14:02:25.617 INFO:tasks.rados.rados.0.smithi017.stdout:93: finishing write tid 3 to smithi017107884-22 2023-03-29T14:02:25.617 INFO:tasks.rados.rados.0.smithi017.stdout:93: finishing write tid 4 to smithi017107884-22 2023-03-29T14:02:25.618 INFO:tasks.rados.rados.0.smithi017.stdout:93: oid 22 updating version 0 to 14 2023-03-29T14:02:25.618 INFO:tasks.rados.rados.0.smithi017.stdout:93: oid 22 version 14 is already newer than 14 2023-03-29T14:02:25.618 INFO:tasks.rados.rados.0.smithi017.stdout:93: oid 22 version 14 is already newer than 14 2023-03-29T14:02:25.618 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 14 (ObjNum 71 snap 3 seq_num 71) dirty exists 2023-03-29T14:02:25.618 INFO:tasks.rados.rados.0.smithi017.stdout:93: left oid 22 (ObjNum 71 snap 3 seq_num 71) 2023-03-29T14:02:25.632 INFO:tasks.rados.rados.0.smithi017.stdout:95: expect (ObjNum 28 snap 0 seq_num 28) 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: finishing write tid 2 to smithi017107884-44 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: finishing write tid 3 to smithi017107884-44 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: finishing write tid 4 to smithi017107884-44 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: oid 44 updating version 0 to 15 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: oid 44 version 15 is already newer than 12 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: oid 44 version 15 is already newer than 15 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 15 (ObjNum 70 snap 3 seq_num 70) dirty exists 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: left oid 44 (ObjNum 70 snap 3 seq_num 70) 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:92: done (4 left) 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:93: done (3 left) 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:94: done (2 left) 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:95: done (1 left) 2023-03-29T14:02:25.729 INFO:tasks.rados.rados.0.smithi017.stdout:96: done (0 left) 2023-03-29T14:02:25.730 INFO:tasks.rados.rados.0.smithi017.stdout:97: snap_remove snap 0 2023-03-29T14:02:25.992 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:25.992 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:25.993 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:26.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:26.121+0000 7f98e9fa7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:26.127+0000 7f98e9fa7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:26.230 INFO:tasks.rados.rados.0.smithi017.stdout:97: done (0 left) 2023-03-29T14:02:26.230 INFO:tasks.rados.rados.0.smithi017.stdout:98: write (excl) oid 43 current snap is 3 2023-03-29T14:02:26.230 INFO:tasks.rados.rados.0.smithi017.stdout:98: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2023-03-29T14:02:26.235 INFO:tasks.rados.rados.0.smithi017.stdout:98: writing smithi017107884-43 from 426911 to 967265 tid 1 2023-03-29T14:02:26.243 INFO:tasks.rados.rados.0.smithi017.stdout:98: writing smithi017107884-43 from 1622009 to 2348692 tid 2 2023-03-29T14:02:26.248 INFO:tasks.rados.rados.0.smithi017.stdout:98: writing smithi017107884-43 from 3121996 to 3226911 tid 3 2023-03-29T14:02:26.248 INFO:tasks.rados.rados.0.smithi017.stdout:99: snap_remove snap 2 2023-03-29T14:02:26.341 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:26.343 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:26.356 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-03-29T14:02:26.228944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231198302,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":324,"num_read_kb":11952,"num_write":730,"num_write_kb":127762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":207388,"kb_used_data":163220,"kb_used_omap":0,"kb_used_meta":22080,"kb_avail":377279972,"statfs":{"total":386547056640,"available":386334691328,"internally_reserved":0,"allocated":167137280,"data_stored":167350016,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":22609920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19395014,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":32,"num_write_kb":8444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014585"},"pg_stats":[{"pgid":"3.f","version":"16'6","reported_seq":24,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:19.690918+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:19.690918+0000","last_peered":"2023-03-29T14:02:19.690918+0000","last_clean":"2023-03-29T14:02:19.690918+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:19.690918+0000","last_undegraded":"2023-03-29T14:02:19.690918+0000","last_fullsized":"2023-03-29T14:02:19.690918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9210104,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"17'3","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.361152+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:20.361152+0000","last_peered":"2023-03-29T14:02:20.361152+0000","last_clean":"2023-03-29T14:02:20.361152+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:20.361152+0000","last_undegraded":"2023-03-29T14:02:20.361152+0000","last_fullsized":"2023-03-29T14:02:20.361152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243718,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'19","reported_seq":56,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:21.144928+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:21.144928+0000","last_peered":"2023-03-29T14:02:21.144928+0000","last_clean":"2023-03-29T14:02:21.144928+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:21.144928+0000","last_undegraded":"2023-03-29T14:02:21.144928+0000","last_fullsized":"2023-03-29T14:02:21.144928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20000066,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":6,"num_write":50,"num_write_kb":13376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"16'16","reported_seq":47,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.970800+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:20.970800+0000","last_peered":"2023-03-29T14:02:20.970800+0000","last_clean":"2023-03-29T14:02:20.970800+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:20.970800+0000","last_undegraded":"2023-03-29T14:02:20.970800+0000","last_fullsized":"2023-03-29T14:02:20.970800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773188,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":5,"num_write":42,"num_write_kb":11360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"17'12","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.818374+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:20.818374+0000","last_peered":"2023-03-29T14:02:20.818374+0000","last_clean":"2023-03-29T14:02:20.818374+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:20.818374+0000","last_undegraded":"2023-03-29T14:02:20.818374+0000","last_fullsized":"2023-03-29T14:02:20.818374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739574,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":5335,"num_write":32,"num_write_kb":9292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:17.500159+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:17.500159+0000","last_peered":"2023-03-29T14:02:17.500159+0000","last_clean":"2023-03-29T14:02:17.500159+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:17.500159+0000","last_undegraded":"2023-03-29T14:02:17.500159+0000","last_fullsized":"2023-03-29T14:02:17.500159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"16'12","reported_seq":38,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.568111+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:20.568111+0000","last_peered":"2023-03-29T14:02:20.568111+0000","last_clean":"2023-03-29T14:02:20.568111+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:20.568111+0000","last_undegraded":"2023-03-29T14:02:20.568111+0000","last_fullsized":"2023-03-29T14:02:20.568111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294172,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":32,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"16'9","reported_seq":31,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.949476+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:20.949476+0000","last_peered":"2023-03-29T14:02:20.949476+0000","last_clean":"2023-03-29T14:02:20.949476+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:20.949476+0000","last_undegraded":"2023-03-29T14:02:20.949476+0000","last_fullsized":"2023-03-29T14:02:20.949476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9983226,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":24,"num_write_kb":6220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:18.393088+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:18.393088+0000","last_peered":"2023-03-29T14:02:18.393088+0000","last_clean":"2023-03-29T14:02:18.393088+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:18.393088+0000","last_undegraded":"2023-03-29T14:02:18.393088+0000","last_fullsized":"2023-03-29T14:02:18.393088+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":24,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.243320+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:20.243320+0000","last_peered":"2023-03-29T14:02:20.243320+0000","last_clean":"2023-03-29T14:02:20.243320+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:20.243320+0000","last_undegraded":"2023-03-29T14:02:20.243320+0000","last_fullsized":"2023-03-29T14:02:20.243320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10184910,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:17.258670+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:17.258670+0000","last_peered":"2023-03-29T14:02:17.258670+0000","last_clean":"2023-03-29T14:02:17.258670+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:17.258670+0000","last_undegraded":"2023-03-29T14:02:17.258670+0000","last_fullsized":"2023-03-29T14:02:17.258670+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"16'6","reported_seq":24,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.331587+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:20.331587+0000","last_peered":"2023-03-29T14:02:20.331587+0000","last_clean":"2023-03-29T14:02:20.331587+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:20.331587+0000","last_undegraded":"2023-03-29T14:02:20.331587+0000","last_fullsized":"2023-03-29T14:02:20.331587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714314,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":3954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:17.238204+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:17.238204+0000","last_peered":"2023-03-29T14:02:17.238204+0000","last_clean":"2023-03-29T14:02:17.238204+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:17.238204+0000","last_undegraded":"2023-03-29T14:02:17.238204+0000","last_fullsized":"2023-03-29T14:02:17.238204+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"17'9","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:21.088413+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:21.088413+0000","last_peered":"2023-03-29T14:02:21.088413+0000","last_clean":"2023-03-29T14:02:21.088413+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:21.088413+0000","last_undegraded":"2023-03-29T14:02:21.088413+0000","last_fullsized":"2023-03-29T14:02:21.088413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16235364,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6448,"num_write":24,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:17.233772+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:17.233772+0000","last_peered":"2023-03-29T14:02:17.233772+0000","last_clean":"2023-03-29T14:02:17.233772+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:17.233772+0000","last_undegraded":"2023-03-29T14:02:17.233772+0000","last_fullsized":"2023-03-29T14:02:17.233772+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":239,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:18.397969+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:18.397969+0000","last_peered":"2023-03-29T14:02:18.397969+0000","last_clean":"2023-03-29T14:02:18.397969+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:18.397969+0000","last_undegraded":"2023-03-29T14:02:18.397969+0000","last_fullsized":"2023-03-29T14:02:18.397969+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"16'12","reported_seq":40,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:21.250286+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:21.250286+0000","last_peered":"2023-03-29T14:02:21.250286+0000","last_clean":"2023-03-29T14:02:21.250286+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:21.250286+0000","last_undegraded":"2023-03-29T14:02:21.250286+0000","last_fullsized":"2023-03-29T14:02:21.250286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13680700,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":4,"num_write":32,"num_write_kb":7906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:17.257905+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:17.257905+0000","last_peered":"2023-03-29T14:02:17.257905+0000","last_clean":"2023-03-29T14:02:17.257905+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:17.257905+0000","last_undegraded":"2023-03-29T14:02:17.257905+0000","last_fullsized":"2023-03-29T14:02:17.257905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"16'25","reported_seq":70,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.839019+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:20.839019+0000","last_peered":"2023-03-29T14:02:20.839019+0000","last_clean":"2023-03-29T14:02:20.839019+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:20.839019+0000","last_undegraded":"2023-03-29T14:02:20.839019+0000","last_fullsized":"2023-03-29T14:02:20.839019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29210170,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":8,"num_write":66,"num_write_kb":17898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:18.401382+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:18.401382+0000","last_peered":"2023-03-29T14:02:18.401382+0000","last_clean":"2023-03-29T14:02:18.401382+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:18.401382+0000","last_undegraded":"2023-03-29T14:02:18.401382+0000","last_fullsized":"2023-03-29T14:02:18.401382+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"17'6","reported_seq":25,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:20.598350+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:20.598350+0000","last_peered":"2023-03-29T14:02:20.598350+0000","last_clean":"2023-03-29T14:02:20.598350+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:20.598350+0000","last_undegraded":"2023-03-29T14:02:20.598350+0000","last_fullsized":"2023-03-29T14:02:20.598350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10588278,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":5196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'19","reported_seq":55,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:21.083166+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:21.083166+0000","last_peered":"2023-03-29T14:02:21.083166+0000","last_clean":"2023-03-29T14:02:21.083166+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:21.083166+0000","last_undegraded":"2023-03-29T14:02:21.083166+0000","last_fullsized":"2023-03-29T14:02:21.083166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243784,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":6,"num_write":50,"num_write_kb":13952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-03-29T14:02:18.394962+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:18.394962+0000","last_peered":"2023-03-29T14:02:18.394962+0000","last_clean":"2023-03-29T14:02:18.394962+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:18.394962+0000","last_undegraded":"2023-03-29T14:02:18.394962+0000","last_fullsized":"2023-03-29T14:02:18.394962+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:17.242186+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:17.242186+0000","last_peered":"2023-03-29T14:02:17.242186+0000","last_clean":"2023-03-29T14:02:17.242186+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:17.242186+0000","last_undegraded":"2023-03-29T14:02:17.242186+0000","last_fullsized":"2023-03-29T14:02:17.242186+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'12","reported_seq":40,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-03-29T14:02:21.206121+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:21.206121+0000","last_peered":"2023-03-29T14:02:21.206121+0000","last_clean":"2023-03-29T14:02:21.206121+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:21.206121+0000","last_undegraded":"2023-03-29T14:02:21.206121+0000","last_fullsized":"2023-03-29T14:02:21.206121+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13915998,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":32,"num_write_kb":8022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":230017566,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":182,"num_read_kb":11832,"num_write":456,"num_write_kb":123974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705679,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56556,"kb_used_data":43948,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94315284,"statfs":{"total":96636764160,"available":96578850816,"internally_reserved":0,"allocated":45002752,"data_stored":44637389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705680,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35932,"kb_used_data":28136,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":28811264,"data_stored":29079388,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705679,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60048,"kb_used_data":47512,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94311792,"statfs":{"total":96636764160,"available":96575275008,"internally_reserved":0,"allocated":48652288,"data_stored":48787972,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705679,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54852,"kb_used_data":43624,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":94316988,"statfs":{"total":96636764160,"available":96580595712,"internally_reserved":0,"allocated":44670976,"data_stored":44845267,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:26.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:26.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:26.848+0000 7f4256f2e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:26.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:26.853+0000 7f4256f2e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:27.235 INFO:tasks.rados.rados.0.smithi017.stdout:99: done (1 left) 2023-03-29T14:02:27.235 INFO:tasks.rados.rados.0.smithi017.stdout:100: write oid 27 current snap is 3 2023-03-29T14:02:27.235 INFO:tasks.rados.rados.0.smithi017.stdout:98: finishing write tid 1 to smithi017107884-43 2023-03-29T14:02:27.235 INFO:tasks.rados.rados.0.smithi017.stdout:100: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2023-03-29T14:02:27.238 INFO:tasks.rados.rados.0.smithi017.stdout:100: writing smithi017107884-27 from 443718 to 1059321 tid 1 2023-03-29T14:02:27.244 INFO:tasks.rados.rados.0.smithi017.stdout:100: writing smithi017107884-27 from 1564138 to 2034479 tid 2 2023-03-29T14:02:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:100: writing smithi017107884-27 from 2633066 to 3243718 tid 3 2023-03-29T14:02:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:101: read oid 34 snap -1 2023-03-29T14:02:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:101: expect (ObjNum 33 snap 0 seq_num 33) 2023-03-29T14:02:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:102: read oid 4 snap -1 2023-03-29T14:02:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:102: expect (ObjNum 56 snap 0 seq_num 56) 2023-03-29T14:02:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:98: finishing write tid 2 to smithi017107884-43 2023-03-29T14:02:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:98: finishing write tid 3 to smithi017107884-43 2023-03-29T14:02:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:103: read oid 19 snap -1 2023-03-29T14:02:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:103: expect (ObjNum 18 snap 0 seq_num 18) 2023-03-29T14:02:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:104: read oid 35 snap -1 2023-03-29T14:02:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:104: expect (ObjNum 34 snap 0 seq_num 34) 2023-03-29T14:02:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:105: read oid 45 snap -1 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:105: expect (ObjNum 61 snap 2 seq_num 61) 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:106: read oid 26 snap -1 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:106: expect (ObjNum 25 snap 0 seq_num 25) 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:107: delete oid 15 current snap is 3 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:98: finishing write tid 4 to smithi017107884-43 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:98: finishing write tid 5 to smithi017107884-43 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:98: oid 43 updating version 0 to 14 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:98: oid 43 version 14 is already newer than 10 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:98: oid 43 version 14 is already newer than 14 2023-03-29T14:02:27.253 INFO:tasks.rados.rados.0.smithi017.stdout:98: oid 43 version 14 is already newer than 14 2023-03-29T14:02:27.254 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 14 (ObjNum 73 snap 3 seq_num 73) dirty exists 2023-03-29T14:02:27.254 INFO:tasks.rados.rados.0.smithi017.stdout:98: left oid 43 (ObjNum 73 snap 3 seq_num 73) 2023-03-29T14:02:27.319 INFO:tasks.rados.rados.0.smithi017.stdout:102: expect (ObjNum 56 snap 0 seq_num 56) 2023-03-29T14:02:27.504 INFO:tasks.rados.rados.0.smithi017.stdout:103: expect (ObjNum 18 snap 0 seq_num 18) 2023-03-29T14:02:27.593 INFO:tasks.rados.rados.0.smithi017.stdout:100: finishing write tid 1 to smithi017107884-27 2023-03-29T14:02:27.593 INFO:tasks.rados.rados.0.smithi017.stdout:100: finishing write tid 2 to smithi017107884-27 2023-03-29T14:02:27.593 INFO:tasks.rados.rados.0.smithi017.stdout:104: expect (ObjNum 34 snap 0 seq_num 34) 2023-03-29T14:02:27.693 INFO:tasks.rados.rados.0.smithi017.stdout:101: expect (ObjNum 33 snap 0 seq_num 33) 2023-03-29T14:02:27.795 INFO:tasks.rados.rados.0.smithi017.stdout:100: finishing write tid 3 to smithi017107884-27 2023-03-29T14:02:27.795 INFO:tasks.rados.rados.0.smithi017.stdout:100: finishing write tid 4 to smithi017107884-27 2023-03-29T14:02:27.795 INFO:tasks.rados.rados.0.smithi017.stdout:100: finishing write tid 5 to smithi017107884-27 2023-03-29T14:02:27.795 INFO:tasks.rados.rados.0.smithi017.stdout:100: oid 27 updating version 0 to 22 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:100: oid 27 version 22 is already newer than 22 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:100: oid 27 version 22 is already newer than 22 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:100: oid 27 version 22 is already newer than 22 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 22 (ObjNum 74 snap 3 seq_num 74) dirty exists 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:100: left oid 27 (ObjNum 74 snap 3 seq_num 74) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:98: done (8 left) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:100: done (7 left) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:101: done (6 left) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:102: done (5 left) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:103: done (4 left) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:104: done (3 left) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:107: done (2 left) 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:108: snap_create 2023-03-29T14:02:27.796 INFO:tasks.rados.rados.0.smithi017.stdout:105: expect (ObjNum 61 snap 2 seq_num 61) 2023-03-29T14:02:27.955 INFO:tasks.rados.rados.0.smithi017.stdout:106: expect (ObjNum 25 snap 0 seq_num 25) 2023-03-29T14:02:28.235 INFO:tasks.rados.rados.0.smithi017.stdout:105: done (2 left) 2023-03-29T14:02:28.236 INFO:tasks.rados.rados.0.smithi017.stdout:106: done (1 left) 2023-03-29T14:02:28.236 INFO:tasks.rados.rados.0.smithi017.stdout:108: done (0 left) 2023-03-29T14:02:28.236 INFO:tasks.rados.rados.0.smithi017.stdout:109: delete oid 42 current snap is 4 2023-03-29T14:02:28.350 INFO:tasks.rados.rados.0.smithi017.stdout:109: done (0 left) 2023-03-29T14:02:28.351 INFO:tasks.rados.rados.0.smithi017.stdout:110: read oid 48 snap 3 2023-03-29T14:02:28.351 INFO:tasks.rados.rados.0.smithi017.stdout:110: expect deleted 2023-03-29T14:02:28.351 INFO:tasks.rados.rados.0.smithi017.stdout:111: snap_create 2023-03-29T14:02:28.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:29.060 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:29.059+0000 7ff860d43700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:29.063 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:29.064+0000 7ff860d43700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:29.238 INFO:tasks.rados.rados.0.smithi017.stdout:110: done (1 left) 2023-03-29T14:02:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:111: done (0 left) 2023-03-29T14:02:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:112: snap_create 2023-03-29T14:02:30.243 INFO:tasks.rados.rados.0.smithi017.stdout:112: done (0 left) 2023-03-29T14:02:30.243 INFO:tasks.rados.rados.0.smithi017.stdout:113: read oid 35 snap -1 2023-03-29T14:02:30.243 INFO:tasks.rados.rados.0.smithi017.stdout:113: expect (ObjNum 34 snap 0 seq_num 34) 2023-03-29T14:02:30.244 INFO:tasks.rados.rados.0.smithi017.stdout:114: read oid 9 snap -1 2023-03-29T14:02:30.244 INFO:tasks.rados.rados.0.smithi017.stdout:114: expect (ObjNum 8 snap 0 seq_num 8) 2023-03-29T14:02:30.244 INFO:tasks.rados.rados.0.smithi017.stdout:115: delete oid 26 current snap is 6 2023-03-29T14:02:30.311 INFO:tasks.rados.rados.0.smithi017.stdout:113: expect (ObjNum 34 snap 0 seq_num 34) 2023-03-29T14:02:30.413 INFO:tasks.rados.rados.0.smithi017.stdout:114: expect (ObjNum 8 snap 0 seq_num 8) 2023-03-29T14:02:30.496 INFO:tasks.rados.rados.0.smithi017.stdout:113: done (2 left) 2023-03-29T14:02:30.496 INFO:tasks.rados.rados.0.smithi017.stdout:114: done (1 left) 2023-03-29T14:02:30.496 INFO:tasks.rados.rados.0.smithi017.stdout:115: done (0 left) 2023-03-29T14:02:30.496 INFO:tasks.rados.rados.0.smithi017.stdout:116: write oid 28 current snap is 6 2023-03-29T14:02:30.496 INFO:tasks.rados.rados.0.smithi017.stdout:116: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2023-03-29T14:02:30.502 INFO:tasks.rados.rados.0.smithi017.stdout:116: writing smithi017107884-28 from 510946 to 1143898 tid 1 2023-03-29T14:02:30.509 INFO:tasks.rados.rados.0.smithi017.stdout:116: writing smithi017107884-28 from 1798066 to 2275745 tid 2 2023-03-29T14:02:30.514 INFO:tasks.rados.rados.0.smithi017.stdout:116: writing smithi017107884-28 from 2742758 to 3199975 tid 3 2023-03-29T14:02:30.514 INFO:tasks.rados.rados.0.smithi017.stdout:117: write oid 50 current snap is 6 2023-03-29T14:02:30.515 INFO:tasks.rados.rados.0.smithi017.stdout:117: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2023-03-29T14:02:30.521 INFO:tasks.rados.rados.0.smithi017.stdout:117: writing smithi017107884-50 from 527753 to 1235954 tid 1 2023-03-29T14:02:30.529 INFO:tasks.rados.rados.0.smithi017.stdout:117: writing smithi017107884-50 from 1856548 to 2594238 tid 2 2023-03-29T14:02:30.533 INFO:tasks.rados.rados.0.smithi017.stdout:117: writing smithi017107884-50 from 3286534 to 3327753 tid 3 2023-03-29T14:02:30.533 INFO:tasks.rados.rados.0.smithi017.stdout:118: write oid 49 current snap is 6 2023-03-29T14:02:30.533 INFO:tasks.rados.rados.0.smithi017.stdout:118: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2023-03-29T14:02:30.540 INFO:tasks.rados.rados.0.smithi017.stdout:118: writing smithi017107884-49 from 544560 to 1328010 tid 1 2023-03-29T14:02:30.546 INFO:tasks.rados.rados.0.smithi017.stdout:118: writing smithi017107884-49 from 1915030 to 2396378 tid 2 2023-03-29T14:02:30.551 INFO:tasks.rados.rados.0.smithi017.stdout:118: writing smithi017107884-49 from 2797604 to 3277365 tid 3 2023-03-29T14:02:30.552 INFO:tasks.rados.rados.0.smithi017.stdout:119: write (excl) oid 26 current snap is 6 2023-03-29T14:02:30.552 INFO:tasks.rados.rados.0.smithi017.stdout:116: finishing write tid 1 to smithi017107884-28 2023-03-29T14:02:30.552 INFO:tasks.rados.rados.0.smithi017.stdout:119: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2023-03-29T14:02:30.557 INFO:tasks.rados.rados.0.smithi017.stdout:119: writing smithi017107884-26 from 561367 to 1020066 tid 1 2023-03-29T14:02:30.564 INFO:tasks.rados.rados.0.smithi017.stdout:119: writing smithi017107884-26 from 1573512 to 2314871 tid 2 2023-03-29T14:02:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:119: writing smithi017107884-26 from 2941380 to 3361367 tid 3 2023-03-29T14:02:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:120: write (excl) oid 44 current snap is 6 2023-03-29T14:02:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:120: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2023-03-29T14:02:30.576 INFO:tasks.rados.rados.0.smithi017.stdout:120: writing smithi017107884-44 from 578174 to 1112122 tid 1 2023-03-29T14:02:30.581 INFO:tasks.rados.rados.0.smithi017.stdout:120: writing smithi017107884-44 from 1515641 to 2000658 tid 2 2023-03-29T14:02:30.588 INFO:tasks.rados.rados.0.smithi017.stdout:120: writing smithi017107884-44 from 2736097 to 3238402 tid 3 2023-03-29T14:02:30.588 INFO:tasks.rados.rados.0.smithi017.stdout:121: read oid 22 snap -1 2023-03-29T14:02:30.588 INFO:tasks.rados.rados.0.smithi017.stdout:121: expect (ObjNum 71 snap 3 seq_num 71) 2023-03-29T14:02:30.588 INFO:tasks.rados.rados.0.smithi017.stdout:122: snap_remove snap 5 2023-03-29T14:02:31.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:31.244 INFO:tasks.rados.rados.0.smithi017.stdout:122: done (6 left) 2023-03-29T14:02:31.244 INFO:tasks.rados.rados.0.smithi017.stdout:123: read oid 37 snap -1 2023-03-29T14:02:31.244 INFO:tasks.rados.rados.0.smithi017.stdout:123: expect (ObjNum 36 snap 0 seq_num 36) 2023-03-29T14:02:31.244 INFO:tasks.rados.rados.0.smithi017.stdout:124: snap_remove snap 4 2023-03-29T14:02:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:31.264+0000 7fe4fdaa0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:31.269+0000 7fe4fdaa0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:31.361 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:31.362 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:31.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:31.490+0000 7fbdc14fb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:31.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:31.496+0000 7fbdc14fb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:31.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:31.711 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-03-29T14:02:31.242998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224610156,"num_objects":122,"num_object_clones":26,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":435,"num_read_kb":81654,"num_write":820,"num_write_kb":154828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":222180,"kb_used_data":198328,"kb_used_omap":0,"kb_used_meta":22656,"kb_avail":377265180,"statfs":{"total":386547056640,"available":386319544320,"internally_reserved":0,"allocated":203087872,"data_stored":216451719,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":23199744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1680700,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":12570,"num_write":18,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010391"},"pg_stats":[{"pgid":"3.f","version":"21'14","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.618490+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:25.618490+0000","last_peered":"2023-03-29T14:02:25.618490+0000","last_clean":"2023-03-29T14:02:25.618490+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:25.618490+0000","last_undegraded":"2023-03-29T14:02:25.618490+0000","last_fullsized":"2023-03-29T14:02:25.618490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10890804,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":5282,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"17'3","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.410207+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:25.410207+0000","last_peered":"2023-03-29T14:02:25.410207+0000","last_clean":"2023-03-29T14:02:25.410207+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:25.410207+0000","last_undegraded":"2023-03-29T14:02:25.410207+0000","last_fullsized":"2023-03-29T14:02:25.410207+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243718,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"21'22","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.556547+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:25.556547+0000","last_peered":"2023-03-29T14:02:25.556547+0000","last_clean":"2023-03-29T14:02:25.556547+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:25.556547+0000","last_undegraded":"2023-03-29T14:02:25.556547+0000","last_fullsized":"2023-03-29T14:02:25.556547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15697540,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":6,"num_write":50,"num_write_kb":13376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"18'21","reported_seq":61,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.282693+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:25.282693+0000","last_peered":"2023-03-29T14:02:25.282693+0000","last_clean":"2023-03-29T14:02:25.282693+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:25.282693+0000","last_undegraded":"2023-03-29T14:02:25.282693+0000","last_fullsized":"2023-03-29T14:02:25.282693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479082,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":6,"num_write":52,"num_write_kb":13740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"19'20","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.391682+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:25.391682+0000","last_peered":"2023-03-29T14:02:25.391682+0000","last_clean":"2023-03-29T14:02:25.391682+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:25.391682+0000","last_undegraded":"2023-03-29T14:02:25.391682+0000","last_fullsized":"2023-03-29T14:02:25.391682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21882450,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5337,"num_write":48,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.285563+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:25.285563+0000","last_peered":"2023-03-29T14:02:25.285563+0000","last_clean":"2023-03-29T14:02:25.285563+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:25.285563+0000","last_undegraded":"2023-03-29T14:02:25.285563+0000","last_fullsized":"2023-03-29T14:02:25.285563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"21'14","reported_seq":45,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.403307+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:25.403307+0000","last_peered":"2023-03-29T14:02:25.403307+0000","last_clean":"2023-03-29T14:02:25.403307+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:25.403307+0000","last_undegraded":"2023-03-29T14:02:25.403307+0000","last_fullsized":"2023-03-29T14:02:25.403307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9579858,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":14,"num_read_kb":4,"num_write":32,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"21'13","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.497431+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:25.497431+0000","last_peered":"2023-03-29T14:02:25.497431+0000","last_clean":"2023-03-29T14:02:25.497431+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:25.497431+0000","last_undegraded":"2023-03-29T14:02:25.497431+0000","last_fullsized":"2023-03-29T14:02:25.497431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12033680,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":7451,"num_write":32,"num_write_kb":9128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.394874+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:25.394874+0000","last_peered":"2023-03-29T14:02:25.394874+0000","last_clean":"2023-03-29T14:02:25.394874+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:25.394874+0000","last_undegraded":"2023-03-29T14:02:25.394874+0000","last_fullsized":"2023-03-29T14:02:25.394874+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.269145+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:25.269145+0000","last_peered":"2023-03-29T14:02:25.269145+0000","last_clean":"2023-03-29T14:02:25.269145+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:25.269145+0000","last_undegraded":"2023-03-29T14:02:25.269145+0000","last_fullsized":"2023-03-29T14:02:25.269145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10184910,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":7292,"num_write":16,"num_write_kb":4726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.267701+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:25.267701+0000","last_peered":"2023-03-29T14:02:25.267701+0000","last_clean":"2023-03-29T14:02:25.267701+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:25.267701+0000","last_undegraded":"2023-03-29T14:02:25.267701+0000","last_fullsized":"2023-03-29T14:02:25.267701+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"16'6","reported_seq":30,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.623541+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:25.623541+0000","last_peered":"2023-03-29T14:02:25.623541+0000","last_clean":"2023-03-29T14:02:25.623541+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:25.623541+0000","last_undegraded":"2023-03-29T14:02:25.623541+0000","last_fullsized":"2023-03-29T14:02:25.623541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714314,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":5878,"num_write":16,"num_write_kb":3954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.290089+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:25.290089+0000","last_peered":"2023-03-29T14:02:25.290089+0000","last_clean":"2023-03-29T14:02:25.290089+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:25.290089+0000","last_undegraded":"2023-03-29T14:02:25.290089+0000","last_fullsized":"2023-03-29T14:02:25.290089+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"17'9","reported_seq":50,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:26.278520+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:26.278520+0000","last_peered":"2023-03-29T14:02:26.278520+0000","last_clean":"2023-03-29T14:02:26.278520+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:26.278520+0000","last_undegraded":"2023-03-29T14:02:26.278520+0000","last_fullsized":"2023-03-29T14:02:26.278520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16235364,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":23597,"num_write":26,"num_write_kb":8796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.289587+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:25.289587+0000","last_peered":"2023-03-29T14:02:25.289587+0000","last_clean":"2023-03-29T14:02:25.289587+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:25.289587+0000","last_undegraded":"2023-03-29T14:02:25.289587+0000","last_fullsized":"2023-03-29T14:02:25.289587+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":243,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.401479+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:25.401479+0000","last_peered":"2023-03-29T14:02:25.401479+0000","last_clean":"2023-03-29T14:02:25.401479+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:25.401479+0000","last_undegraded":"2023-03-29T14:02:25.401479+0000","last_fullsized":"2023-03-29T14:02:25.401479+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"21'18","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.449229+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:25.449229+0000","last_peered":"2023-03-29T14:02:25.449229+0000","last_clean":"2023-03-29T14:02:25.449229+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:25.449229+0000","last_undegraded":"2023-03-29T14:02:25.449229+0000","last_fullsized":"2023-03-29T14:02:25.449229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15731154,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":10325,"num_write":42,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.288459+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:25.288459+0000","last_peered":"2023-03-29T14:02:25.288459+0000","last_clean":"2023-03-29T14:02:25.288459+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:25.288459+0000","last_undegraded":"2023-03-29T14:02:25.288459+0000","last_fullsized":"2023-03-29T14:02:25.288459+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"20'32","reported_seq":85,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-03-29T14:02:24.708477+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:24.708477+0000","last_peered":"2023-03-29T14:02:24.708477+0000","last_clean":"2023-03-29T14:02:24.708477+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:24.708477+0000","last_undegraded":"2023-03-29T14:02:24.708477+0000","last_fullsized":"2023-03-29T14:02:24.708477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24100908,"num_objects":16,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":34,"num_read_kb":5289,"num_write":74,"num_write_kb":20506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.397688+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:25.397688+0000","last_peered":"2023-03-29T14:02:25.397688+0000","last_clean":"2023-03-29T14:02:25.397688+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:25.397688+0000","last_undegraded":"2023-03-29T14:02:25.397688+0000","last_fullsized":"2023-03-29T14:02:25.397688+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"21'15","reported_seq":45,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.663866+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:25.663866+0000","last_peered":"2023-03-29T14:02:25.663866+0000","last_clean":"2023-03-29T14:02:25.663866+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:25.663866+0000","last_undegraded":"2023-03-29T14:02:25.663866+0000","last_fullsized":"2023-03-29T14:02:25.663866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11495856,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":4,"num_write":34,"num_write_kb":10216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'19","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.408819+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:25.408819+0000","last_peered":"2023-03-29T14:02:25.408819+0000","last_clean":"2023-03-29T14:02:25.408819+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:25.408819+0000","last_undegraded":"2023-03-29T14:02:25.408819+0000","last_fullsized":"2023-03-29T14:02:25.408819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243784,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":6072,"num_write":50,"num_write_kb":13952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.383330+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:25.383330+0000","last_peered":"2023-03-29T14:02:25.383330+0000","last_clean":"2023-03-29T14:02:25.383330+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:25.383330+0000","last_undegraded":"2023-03-29T14:02:25.383330+0000","last_fullsized":"2023-03-29T14:02:25.383330+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-03-29T14:02:25.291325+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:25.291325+0000","last_peered":"2023-03-29T14:02:25.291325+0000","last_clean":"2023-03-29T14:02:25.291325+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:25.291325+0000","last_undegraded":"2023-03-29T14:02:25.291325+0000","last_fullsized":"2023-03-29T14:02:25.291325+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'13","reported_seq":46,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-03-29T14:02:24.267318+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:24.267318+0000","last_peered":"2023-03-29T14:02:24.267318+0000","last_clean":"2023-03-29T14:02:24.267318+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:24.267318+0000","last_undegraded":"2023-03-29T14:02:24.267318+0000","last_fullsized":"2023-03-29T14:02:24.267318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13915998,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":4990,"num_write":32,"num_write_kb":8022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":223429420,"num_objects":118,"num_object_clones":26,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":293,"num_read_kb":81534,"num_write":546,"num_write_kb":151040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61232,"kb_used_data":55396,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94310608,"statfs":{"total":96636764160,"available":96574062592,"internally_reserved":0,"allocated":56725504,"data_stored":60634881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40304,"kb_used_data":34276,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":94331536,"statfs":{"total":96636764160,"available":96595492864,"internally_reserved":0,"allocated":35098624,"data_stored":36911667,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705684,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64192,"kb_used_data":57972,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":94307648,"statfs":{"total":96636764160,"available":96571031552,"internally_reserved":0,"allocated":59363328,"data_stored":63565555,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705684,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56452,"kb_used_data":50684,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94315388,"statfs":{"total":96636764160,"available":96578957312,"internally_reserved":0,"allocated":51900416,"data_stored":55339616,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:32.250 INFO:tasks.rados.rados.0.smithi017.stdout:124: done (7 left) 2023-03-29T14:02:32.250 INFO:tasks.rados.rados.0.smithi017.stdout:125: snap_remove snap 3 2023-03-29T14:02:33.253 INFO:tasks.rados.rados.0.smithi017.stdout:125: done (7 left) 2023-03-29T14:02:33.253 INFO:tasks.rados.rados.0.smithi017.stdout:126: write oid 36 current snap is 6 2023-03-29T14:02:33.254 INFO:tasks.rados.rados.0.smithi017.stdout:126: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2023-03-29T14:02:33.257 INFO:tasks.rados.rados.0.smithi017.stdout:126: writing smithi017107884-36 from 594981 to 1204178 tid 1 2023-03-29T14:02:33.266 INFO:tasks.rados.rados.0.smithi017.stdout:126: writing smithi017107884-36 from 1974123 to 2719151 tid 2 2023-03-29T14:02:33.270 INFO:tasks.rados.rados.0.smithi017.stdout:126: writing smithi017107884-36 from 3279873 to 3394981 tid 3 2023-03-29T14:02:33.270 INFO:tasks.rados.rados.0.smithi017.stdout:127: write (excl) oid 35 current snap is 6 2023-03-29T14:02:33.270 INFO:tasks.rados.rados.0.smithi017.stdout:116: finishing write tid 2 to smithi017107884-28 2023-03-29T14:02:33.271 INFO:tasks.rados.rados.0.smithi017.stdout:127: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2023-03-29T14:02:33.275 INFO:tasks.rados.rados.0.smithi017.stdout:127: writing smithi017107884-35 from 611788 to 1012587 tid 1 2023-03-29T14:02:33.282 INFO:tasks.rados.rados.0.smithi017.stdout:127: writing smithi017107884-35 from 1748958 to 2237644 tid 2 2023-03-29T14:02:33.289 INFO:tasks.rados.rados.0.smithi017.stdout:127: writing smithi017107884-35 from 2907296 to 3411788 tid 3 2023-03-29T14:02:33.289 INFO:tasks.rados.rados.0.smithi017.stdout:128: write oid 2 current snap is 6 2023-03-29T14:02:33.289 INFO:tasks.rados.rados.0.smithi017.stdout:128: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2023-03-29T14:02:33.295 INFO:tasks.rados.rados.0.smithi017.stdout:128: writing smithi017107884-2 from 628595 to 1104643 tid 1 2023-03-29T14:02:33.302 INFO:tasks.rados.rados.0.smithi017.stdout:128: writing smithi017107884-2 from 1807440 to 2439784 tid 2 2023-03-29T14:02:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:128: writing smithi017107884-2 from 2934719 to 3387193 tid 3 2023-03-29T14:02:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:129: read oid 9 snap -1 2023-03-29T14:02:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:129: expect (ObjNum 8 snap 0 seq_num 8) 2023-03-29T14:02:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:130: read oid 46 snap -1 2023-03-29T14:02:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:130: expect (ObjNum 62 snap 2 seq_num 62) 2023-03-29T14:02:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:131: write oid 1 current snap is 6 2023-03-29T14:02:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:131: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2023-03-29T14:02:33.314 INFO:tasks.rados.rados.0.smithi017.stdout:131: writing smithi017107884-1 from 645402 to 1196699 tid 1 2023-03-29T14:02:33.320 INFO:tasks.rados.rados.0.smithi017.stdout:131: writing smithi017107884-1 from 1749569 to 2241924 tid 2 2023-03-29T14:02:33.327 INFO:tasks.rados.rados.0.smithi017.stdout:131: writing smithi017107884-1 from 2845789 to 3445402 tid 3 2023-03-29T14:02:33.327 INFO:tasks.rados.rados.0.smithi017.stdout:132: read oid 19 snap -1 2023-03-29T14:02:33.327 INFO:tasks.rados.rados.0.smithi017.stdout:132: expect (ObjNum 18 snap 0 seq_num 18) 2023-03-29T14:02:33.327 INFO:tasks.rados.rados.0.smithi017.stdout:133: write oid 43 current snap is 6 2023-03-29T14:02:33.327 INFO:tasks.rados.rados.0.smithi017.stdout:121: expect (ObjNum 71 snap 3 seq_num 71) 2023-03-29T14:02:33.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:33.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:33.478+0000 7fc275af4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:33.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:33.483+0000 7fc275af4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:33.619 INFO:tasks.rados.rados.0.smithi017.stdout:116: finishing write tid 3 to smithi017107884-28 2023-03-29T14:02:33.619 INFO:tasks.rados.rados.0.smithi017.stdout:116: finishing write tid 4 to smithi017107884-28 2023-03-29T14:02:33.619 INFO:tasks.rados.rados.0.smithi017.stdout:116: finishing write tid 5 to smithi017107884-28 2023-03-29T14:02:33.619 INFO:tasks.rados.rados.0.smithi017.stdout:116: oid 28 updating version 0 to 18 2023-03-29T14:02:33.619 INFO:tasks.rados.rados.0.smithi017.stdout:116: oid 28 version 18 is already newer than 18 2023-03-29T14:02:33.619 INFO:tasks.rados.rados.0.smithi017.stdout:116: oid 28 version 18 is already newer than 18 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:116: oid 28 version 18 is already newer than 18 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 18 (ObjNum 78 snap 6 seq_num 78) dirty exists 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:116: left oid 28 (ObjNum 78 snap 6 seq_num 78) 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:119: finishing write tid 1 to smithi017107884-26 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:117: finishing write tid 1 to smithi017107884-50 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:118: finishing write tid 1 to smithi017107884-49 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:118: finishing write tid 2 to smithi017107884-49 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:118: finishing write tid 3 to smithi017107884-49 2023-03-29T14:02:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:118: finishing write tid 4 to smithi017107884-49 2023-03-29T14:02:33.621 INFO:tasks.rados.rados.0.smithi017.stdout:133: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2023-03-29T14:02:33.626 INFO:tasks.rados.rados.0.smithi017.stdout:133: writing smithi017107884-43 from 662209 to 1288755 tid 1 2023-03-29T14:02:33.632 INFO:tasks.rados.rados.0.smithi017.stdout:133: writing smithi017107884-43 from 1808051 to 2444064 tid 2 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:133: writing smithi017107884-43 from 2873212 to 3462209 tid 3 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:116: done (14 left) 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:121: done (13 left) 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:134: snap_create 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:118: finishing write tid 5 to smithi017107884-49 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:118: oid 49 updating version 0 to 37 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:118: oid 49 version 37 is already newer than 37 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:118: oid 49 version 37 is already newer than 37 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:118: oid 49 version 37 is already newer than 37 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 37 (ObjNum 80 snap 6 seq_num 80) dirty exists 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:118: left oid 49 (ObjNum 80 snap 6 seq_num 80) 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:119: finishing write tid 2 to smithi017107884-26 2023-03-29T14:02:33.640 INFO:tasks.rados.rados.0.smithi017.stdout:117: finishing write tid 2 to smithi017107884-50 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: finishing write tid 3 to smithi017107884-50 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: finishing write tid 4 to smithi017107884-50 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:120: finishing write tid 1 to smithi017107884-44 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: finishing write tid 5 to smithi017107884-50 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: oid 50 updating version 0 to 19 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: oid 50 version 19 is already newer than 19 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: oid 50 version 19 is already newer than 19 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: oid 50 version 19 is already newer than 19 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 19 (ObjNum 79 snap 6 seq_num 79) dirty exists 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:117: left oid 50 (ObjNum 79 snap 6 seq_num 79) 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:119: finishing write tid 3 to smithi017107884-26 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:119: finishing write tid 4 to smithi017107884-26 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:119: finishing write tid 5 to smithi017107884-26 2023-03-29T14:02:33.641 INFO:tasks.rados.rados.0.smithi017.stdout:119: oid 26 updating version 0 to 28 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:119: oid 26 version 28 is already newer than 25 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:119: oid 26 version 28 is already newer than 28 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:119: oid 26 version 28 is already newer than 28 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 28 (ObjNum 81 snap 6 seq_num 81) dirty exists 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:119: left oid 26 (ObjNum 81 snap 6 seq_num 81) 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: finishing write tid 2 to smithi017107884-44 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: finishing write tid 3 to smithi017107884-44 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: finishing write tid 4 to smithi017107884-44 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: finishing write tid 5 to smithi017107884-44 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: oid 44 updating version 0 to 20 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: oid 44 version 20 is already newer than 20 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: oid 44 version 20 is already newer than 20 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: oid 44 version 20 is already newer than 16 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 20 (ObjNum 82 snap 6 seq_num 82) dirty exists 2023-03-29T14:02:33.642 INFO:tasks.rados.rados.0.smithi017.stdout:120: left oid 44 (ObjNum 82 snap 6 seq_num 82) 2023-03-29T14:02:33.643 INFO:tasks.rados.rados.0.smithi017.stdout:123: expect (ObjNum 36 snap 0 seq_num 36) 2023-03-29T14:02:33.740 INFO:tasks.rados.rados.0.smithi017.stdout:129: expect (ObjNum 8 snap 0 seq_num 8) 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: finishing write tid 1 to smithi017107884-2 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: finishing write tid 2 to smithi017107884-2 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: finishing write tid 3 to smithi017107884-2 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:126: finishing write tid 1 to smithi017107884-36 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: finishing write tid 4 to smithi017107884-2 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: finishing write tid 5 to smithi017107884-2 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: oid 2 updating version 0 to 41 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: oid 2 version 41 is already newer than 41 2023-03-29T14:02:33.831 INFO:tasks.rados.rados.0.smithi017.stdout:128: oid 2 version 41 is already newer than 39 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:128: oid 2 version 41 is already newer than 41 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 41 (ObjNum 85 snap 6 seq_num 85) dirty exists 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:128: left oid 2 (ObjNum 85 snap 6 seq_num 85) 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:126: finishing write tid 2 to smithi017107884-36 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: finishing write tid 1 to smithi017107884-1 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: finishing write tid 2 to smithi017107884-1 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: finishing write tid 3 to smithi017107884-1 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: finishing write tid 4 to smithi017107884-1 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: finishing write tid 5 to smithi017107884-1 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: oid 1 updating version 0 to 17 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: oid 1 version 17 is already newer than 17 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: oid 1 version 17 is already newer than 17 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: oid 1 version 17 is already newer than 17 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 17 (ObjNum 86 snap 6 seq_num 86) dirty exists 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:131: left oid 1 (ObjNum 86 snap 6 seq_num 86) 2023-03-29T14:02:33.832 INFO:tasks.rados.rados.0.smithi017.stdout:130: expect (ObjNum 62 snap 2 seq_num 62) 2023-03-29T14:02:33.991 INFO:tasks.rados.rados.0.smithi017.stdout:126: finishing write tid 3 to smithi017107884-36 2023-03-29T14:02:33.991 INFO:tasks.rados.rados.0.smithi017.stdout:132: expect (ObjNum 18 snap 0 seq_num 18) 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:126: finishing write tid 4 to smithi017107884-36 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:126: finishing write tid 5 to smithi017107884-36 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:126: oid 36 updating version 0 to 23 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:126: oid 36 version 23 is already newer than 23 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:126: oid 36 version 23 is already newer than 23 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:126: oid 36 version 23 is already newer than 23 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 23 (ObjNum 83 snap 6 seq_num 83) dirty exists 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:126: left oid 36 (ObjNum 83 snap 6 seq_num 83) 2023-03-29T14:02:34.079 INFO:tasks.rados.rados.0.smithi017.stdout:127: finishing write tid 1 to smithi017107884-35 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: finishing write tid 1 to smithi017107884-43 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: finishing write tid 2 to smithi017107884-43 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: finishing write tid 3 to smithi017107884-43 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: finishing write tid 4 to smithi017107884-43 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:127: finishing write tid 2 to smithi017107884-35 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: finishing write tid 5 to smithi017107884-43 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: oid 43 updating version 0 to 27 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: oid 43 version 27 is already newer than 27 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: oid 43 version 27 is already newer than 27 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: oid 43 version 27 is already newer than 27 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 27 (ObjNum 87 snap 6 seq_num 87) dirty exists 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:133: left oid 43 (ObjNum 87 snap 6 seq_num 87) 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:127: finishing write tid 3 to smithi017107884-35 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:127: finishing write tid 4 to smithi017107884-35 2023-03-29T14:02:34.080 INFO:tasks.rados.rados.0.smithi017.stdout:127: finishing write tid 5 to smithi017107884-35 2023-03-29T14:02:34.081 INFO:tasks.rados.rados.0.smithi017.stdout:127: oid 35 updating version 0 to 24 2023-03-29T14:02:34.081 INFO:tasks.rados.rados.0.smithi017.stdout:127: oid 35 version 24 is already newer than 21 2023-03-29T14:02:34.081 INFO:tasks.rados.rados.0.smithi017.stdout:127: oid 35 version 24 is already newer than 24 2023-03-29T14:02:34.081 INFO:tasks.rados.rados.0.smithi017.stdout:127: oid 35 version 24 is already newer than 24 2023-03-29T14:02:34.081 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 24 (ObjNum 84 snap 6 seq_num 84) dirty exists 2023-03-29T14:02:34.081 INFO:tasks.rados.rados.0.smithi017.stdout:127: left oid 35 (ObjNum 84 snap 6 seq_num 84) 2023-03-29T14:02:34.257 INFO:tasks.rados.rados.0.smithi017.stdout:117: done (13 left) 2023-03-29T14:02:34.257 INFO:tasks.rados.rados.0.smithi017.stdout:118: done (12 left) 2023-03-29T14:02:34.257 INFO:tasks.rados.rados.0.smithi017.stdout:119: done (11 left) 2023-03-29T14:02:34.257 INFO:tasks.rados.rados.0.smithi017.stdout:120: done (10 left) 2023-03-29T14:02:34.257 INFO:tasks.rados.rados.0.smithi017.stdout:123: done (9 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:126: done (8 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:127: done (7 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:128: done (6 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:129: done (5 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:130: done (4 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:131: done (3 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:132: done (2 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:133: done (1 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:134: done (0 left) 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:135: write (excl) oid 42 current snap is 7 2023-03-29T14:02:34.258 INFO:tasks.rados.rados.0.smithi017.stdout:135: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2023-03-29T14:02:34.264 INFO:tasks.rados.rados.0.smithi017.stdout:135: writing smithi017107884-42 from 679016 to 1380811 tid 1 2023-03-29T14:02:34.269 INFO:tasks.rados.rados.0.smithi017.stdout:135: writing smithi017107884-42 from 1866533 to 2362557 tid 2 2023-03-29T14:02:34.274 INFO:tasks.rados.rados.0.smithi017.stdout:135: writing smithi017107884-42 from 2900635 to 3303278 tid 3 2023-03-29T14:02:34.275 INFO:tasks.rados.rados.0.smithi017.stdout:136: snap_remove snap 6 2023-03-29T14:02:35.261 INFO:tasks.rados.rados.0.smithi017.stdout:136: done (1 left) 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:137: read oid 2 snap -1 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:137: expect (ObjNum 85 snap 6 seq_num 85) 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:138: read oid 16 snap -1 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:138: expect (ObjNum 15 snap 0 seq_num 15) 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:139: read oid 32 snap -1 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:139: expect (ObjNum 31 snap 0 seq_num 31) 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:140: read oid 22 snap -1 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:140: expect (ObjNum 71 snap 3 seq_num 71) 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:141: write oid 17 current snap is 7 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:135: finishing write tid 1 to smithi017107884-42 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:135: finishing write tid 2 to smithi017107884-42 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:135: finishing write tid 3 to smithi017107884-42 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:135: finishing write tid 4 to smithi017107884-42 2023-03-29T14:02:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:135: finishing write tid 5 to smithi017107884-42 2023-03-29T14:02:35.263 INFO:tasks.rados.rados.0.smithi017.stdout:135: oid 42 updating version 0 to 27 2023-03-29T14:02:35.263 INFO:tasks.rados.rados.0.smithi017.stdout:135: oid 42 version 27 is already newer than 27 2023-03-29T14:02:35.263 INFO:tasks.rados.rados.0.smithi017.stdout:135: oid 42 version 27 is already newer than 27 2023-03-29T14:02:35.263 INFO:tasks.rados.rados.0.smithi017.stdout:135: oid 42 version 27 is already newer than 24 2023-03-29T14:02:35.263 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 27 (ObjNum 88 snap 7 seq_num 88) dirty exists 2023-03-29T14:02:35.263 INFO:tasks.rados.rados.0.smithi017.stdout:135: left oid 42 (ObjNum 88 snap 7 seq_num 88) 2023-03-29T14:02:35.263 INFO:tasks.rados.rados.0.smithi017.stdout:141: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2023-03-29T14:02:35.269 INFO:tasks.rados.rados.0.smithi017.stdout:141: writing smithi017107884-17 from 695823 to 1472867 tid 1 2023-03-29T14:02:35.275 INFO:tasks.rados.rados.0.smithi017.stdout:141: writing smithi017107884-17 from 1925015 to 2564697 tid 2 2023-03-29T14:02:35.280 INFO:tasks.rados.rados.0.smithi017.stdout:141: writing smithi017107884-17 from 3328058 to 3495823 tid 3 2023-03-29T14:02:35.280 INFO:tasks.rados.rados.0.smithi017.stdout:135: done (5 left) 2023-03-29T14:02:35.281 INFO:tasks.rados.rados.0.smithi017.stdout:142: write (excl) oid 29 current snap is 7 2023-03-29T14:02:35.281 INFO:tasks.rados.rados.0.smithi017.stdout:142: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2023-03-29T14:02:35.286 INFO:tasks.rados.rados.0.smithi017.stdout:142: writing smithi017107884-29 from 712630 to 1164923 tid 1 2023-03-29T14:02:35.292 INFO:tasks.rados.rados.0.smithi017.stdout:142: writing smithi017107884-29 from 1583497 to 2083190 tid 2 2023-03-29T14:02:35.296 INFO:tasks.rados.rados.0.smithi017.stdout:142: writing smithi017107884-29 from 2555481 to 2980668 tid 3 2023-03-29T14:02:35.297 INFO:tasks.rados.rados.0.smithi017.stdout:143: write (excl) oid 3 current snap is 7 2023-03-29T14:02:35.297 INFO:tasks.rados.rados.0.smithi017.stdout:143: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2023-03-29T14:02:35.305 INFO:tasks.rados.rados.0.smithi017.stdout:143: writing smithi017107884-3 from 729437 to 1256979 tid 1 2023-03-29T14:02:35.311 INFO:tasks.rados.rados.0.smithi017.stdout:143: writing smithi017107884-3 from 1925626 to 2568977 tid 2 2023-03-29T14:02:35.316 INFO:tasks.rados.rados.0.smithi017.stdout:143: writing smithi017107884-3 from 3266551 to 3529437 tid 3 2023-03-29T14:02:35.317 INFO:tasks.rados.rados.0.smithi017.stdout:144: write oid 14 current snap is 7 2023-03-29T14:02:35.317 INFO:tasks.rados.rados.0.smithi017.stdout:144: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2023-03-29T14:02:35.324 INFO:tasks.rados.rados.0.smithi017.stdout:144: writing smithi017107884-14 from 746244 to 1465388 tid 1 2023-03-29T14:02:35.331 INFO:tasks.rados.rados.0.smithi017.stdout:144: writing smithi017107884-14 from 2100461 to 2603823 tid 2 2023-03-29T14:02:35.336 INFO:tasks.rados.rados.0.smithi017.stdout:144: writing smithi017107884-14 from 3126680 to 3546244 tid 3 2023-03-29T14:02:35.336 INFO:tasks.rados.rados.0.smithi017.stdout:145: write oid 13 current snap is 7 2023-03-29T14:02:35.336 INFO:tasks.rados.rados.0.smithi017.stdout:145: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2023-03-29T14:02:35.345 INFO:tasks.rados.rados.0.smithi017.stdout:145: writing smithi017107884-13 from 763051 to 1557444 tid 1 2023-03-29T14:02:35.352 INFO:tasks.rados.rados.0.smithi017.stdout:145: writing smithi017107884-13 from 2158943 to 2805963 tid 2 2023-03-29T14:02:35.357 INFO:tasks.rados.rados.0.smithi017.stdout:145: writing smithi017107884-13 from 3437750 to 3563051 tid 3 2023-03-29T14:02:35.357 INFO:tasks.rados.rados.0.smithi017.stdout:146: write oid 34 current snap is 7 2023-03-29T14:02:35.357 INFO:tasks.rados.rados.0.smithi017.stdout:146: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2023-03-29T14:02:35.364 INFO:tasks.rados.rados.0.smithi017.stdout:146: writing smithi017107884-34 from 779858 to 1249500 tid 1 2023-03-29T14:02:35.371 INFO:tasks.rados.rados.0.smithi017.stdout:146: writing smithi017107884-34 from 1817425 to 2324456 tid 2 2023-03-29T14:02:35.377 INFO:tasks.rados.rados.0.smithi017.stdout:146: writing smithi017107884-34 from 2781526 to 3368154 tid 3 2023-03-29T14:02:35.377 INFO:tasks.rados.rados.0.smithi017.stdout:147: read oid 35 snap -1 2023-03-29T14:02:35.378 INFO:tasks.rados.rados.0.smithi017.stdout:147: expect (ObjNum 84 snap 6 seq_num 84) 2023-03-29T14:02:35.378 INFO:tasks.rados.rados.0.smithi017.stdout:148: read oid 47 snap -1 2023-03-29T14:02:35.378 INFO:tasks.rados.rados.0.smithi017.stdout:148: expect (ObjNum 46 snap 0 seq_num 46) 2023-03-29T14:02:35.378 INFO:tasks.rados.rados.0.smithi017.stdout:149: delete oid 49 current snap is 7 2023-03-29T14:02:35.378 INFO:tasks.rados.rados.0.smithi017.stdout:138: expect (ObjNum 15 snap 0 seq_num 15) 2023-03-29T14:02:35.472 INFO:tasks.rados.rados.0.smithi017.stdout:137: expect (ObjNum 85 snap 6 seq_num 85) 2023-03-29T14:02:35.718 INFO:tasks.rados.rados.0.smithi017.stdout:140: expect (ObjNum 71 snap 3 seq_num 71) 2023-03-29T14:02:35.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:35.936 INFO:tasks.rados.rados.0.smithi017.stdout:142: finishing write tid 1 to smithi017107884-29 2023-03-29T14:02:35.936 INFO:tasks.rados.rados.0.smithi017.stdout:141: finishing write tid 1 to smithi017107884-17 2023-03-29T14:02:35.936 INFO:tasks.rados.rados.0.smithi017.stdout:141: finishing write tid 2 to smithi017107884-17 2023-03-29T14:02:35.936 INFO:tasks.rados.rados.0.smithi017.stdout:139: expect (ObjNum 31 snap 0 seq_num 31) 2023-03-29T14:02:35.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:35.949+0000 7f55e0eea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:35.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:35.954+0000 7f55e0eea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: finishing write tid 3 to smithi017107884-17 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: finishing write tid 4 to smithi017107884-17 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: finishing write tid 5 to smithi017107884-17 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: oid 17 updating version 0 to 26 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: oid 17 version 26 is already newer than 26 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: oid 17 version 26 is already newer than 26 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: oid 17 version 26 is already newer than 26 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 26 (ObjNum 89 snap 7 seq_num 89) dirty exists 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:141: left oid 17 (ObjNum 89 snap 7 seq_num 89) 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:143: finishing write tid 1 to smithi017107884-3 2023-03-29T14:02:36.044 INFO:tasks.rados.rados.0.smithi017.stdout:144: finishing write tid 1 to smithi017107884-14 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:144: finishing write tid 2 to smithi017107884-14 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:142: finishing write tid 2 to smithi017107884-29 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: finishing write tid 2 to smithi017107884-3 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: finishing write tid 3 to smithi017107884-3 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: finishing write tid 4 to smithi017107884-3 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:142: finishing write tid 3 to smithi017107884-29 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:144: finishing write tid 3 to smithi017107884-14 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: finishing write tid 5 to smithi017107884-3 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: oid 3 updating version 0 to 42 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: oid 3 updating version 42 to 45 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: oid 3 version 45 is already newer than 45 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: oid 3 version 45 is already newer than 45 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 45 (ObjNum 91 snap 7 seq_num 91) dirty exists 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:143: left oid 3 (ObjNum 91 snap 7 seq_num 91) 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:142: finishing write tid 4 to smithi017107884-29 2023-03-29T14:02:36.045 INFO:tasks.rados.rados.0.smithi017.stdout:142: finishing write tid 5 to smithi017107884-29 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:142: oid 29 updating version 0 to 10 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:142: oid 29 version 10 is already newer than 10 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:142: oid 29 version 10 is already newer than 7 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:142: oid 29 version 10 is already newer than 10 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 10 (ObjNum 90 snap 7 seq_num 90) dirty exists 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:142: left oid 29 (ObjNum 90 snap 7 seq_num 90) 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:144: finishing write tid 4 to smithi017107884-14 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:144: finishing write tid 5 to smithi017107884-14 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:144: oid 14 updating version 0 to 18 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:144: oid 14 version 18 is already newer than 18 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:144: oid 14 version 18 is already newer than 18 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:144: oid 14 version 18 is already newer than 18 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 18 (ObjNum 92 snap 7 seq_num 92) dirty exists 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:144: left oid 14 (ObjNum 92 snap 7 seq_num 92) 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:145: finishing write tid 1 to smithi017107884-13 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:145: finishing write tid 2 to smithi017107884-13 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:145: finishing write tid 3 to smithi017107884-13 2023-03-29T14:02:36.046 INFO:tasks.rados.rados.0.smithi017.stdout:145: finishing write tid 4 to smithi017107884-13 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:145: finishing write tid 5 to smithi017107884-13 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:145: oid 13 updating version 0 to 23 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:145: oid 13 version 23 is already newer than 23 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:145: oid 13 version 23 is already newer than 23 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:145: oid 13 version 23 is already newer than 23 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 23 (ObjNum 93 snap 7 seq_num 93) dirty exists 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:145: left oid 13 (ObjNum 93 snap 7 seq_num 93) 2023-03-29T14:02:36.047 INFO:tasks.rados.rados.0.smithi017.stdout:147: expect (ObjNum 84 snap 6 seq_num 84) 2023-03-29T14:02:36.216 INFO:tasks.rados.rados.0.smithi017.stdout:146: finishing write tid 1 to smithi017107884-34 2023-03-29T14:02:36.216 INFO:tasks.rados.rados.0.smithi017.stdout:146: finishing write tid 2 to smithi017107884-34 2023-03-29T14:02:36.216 INFO:tasks.rados.rados.0.smithi017.stdout:146: finishing write tid 3 to smithi017107884-34 2023-03-29T14:02:36.216 INFO:tasks.rados.rados.0.smithi017.stdout:148: expect (ObjNum 46 snap 0 seq_num 46) 2023-03-29T14:02:36.324 INFO:tasks.rados.rados.0.smithi017.stdout:146: finishing write tid 4 to smithi017107884-34 2023-03-29T14:02:36.324 INFO:tasks.rados.rados.0.smithi017.stdout:146: finishing write tid 5 to smithi017107884-34 2023-03-29T14:02:36.324 INFO:tasks.rados.rados.0.smithi017.stdout:146: oid 34 updating version 0 to 24 2023-03-29T14:02:36.324 INFO:tasks.rados.rados.0.smithi017.stdout:146: oid 34 version 24 is already newer than 24 2023-03-29T14:02:36.324 INFO:tasks.rados.rados.0.smithi017.stdout:146: oid 34 version 24 is already newer than 24 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:146: oid 34 version 24 is already newer than 24 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 24 (ObjNum 94 snap 7 seq_num 94) dirty exists 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:146: left oid 34 (ObjNum 94 snap 7 seq_num 94) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:137: done (12 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:138: done (11 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:139: done (10 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:140: done (9 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:141: done (8 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:142: done (7 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:143: done (6 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:144: done (5 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:145: done (4 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:146: done (3 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:147: done (2 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:148: done (1 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:149: done (0 left) 2023-03-29T14:02:36.325 INFO:tasks.rados.rados.0.smithi017.stdout:150: write (excl) oid 1 current snap is 7 2023-03-29T14:02:36.326 INFO:tasks.rados.rados.0.smithi017.stdout:150: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2023-03-29T14:02:36.329 INFO:tasks.rados.rados.0.smithi017.stdout:150: writing smithi017107884-1 from 413472 to 1033612 tid 1 2023-03-29T14:02:36.336 INFO:tasks.rados.rados.0.smithi017.stdout:150: writing smithi017107884-1 from 1818036 to 2328736 tid 2 2023-03-29T14:02:36.344 INFO:tasks.rados.rados.0.smithi017.stdout:150: writing smithi017107884-1 from 3120019 to 3613472 tid 3 2023-03-29T14:02:36.344 INFO:tasks.rados.rados.0.smithi017.stdout:151: delete oid 19 current snap is 7 2023-03-29T14:02:36.378 INFO:tasks.rados.rados.0.smithi017.stdout:150: finishing write tid 1 to smithi017107884-1 2023-03-29T14:02:36.406 INFO:tasks.rados.rados.0.smithi017.stdout:150: finishing write tid 2 to smithi017107884-1 2023-03-29T14:02:36.408 INFO:tasks.rados.rados.0.smithi017.stdout:150: finishing write tid 3 to smithi017107884-1 2023-03-29T14:02:36.408 INFO:tasks.rados.rados.0.smithi017.stdout:150: finishing write tid 4 to smithi017107884-1 2023-03-29T14:02:36.411 INFO:tasks.rados.rados.0.smithi017.stdout:150: finishing write tid 5 to smithi017107884-1 2023-03-29T14:02:36.411 INFO:tasks.rados.rados.0.smithi017.stdout:150: oid 1 updating version 0 to 21 2023-03-29T14:02:36.411 INFO:tasks.rados.rados.0.smithi017.stdout:150: oid 1 version 21 is already newer than 18 2023-03-29T14:02:36.411 INFO:tasks.rados.rados.0.smithi017.stdout:150: oid 1 version 21 is already newer than 21 2023-03-29T14:02:36.411 INFO:tasks.rados.rados.0.smithi017.stdout:150: oid 1 version 21 is already newer than 21 2023-03-29T14:02:36.411 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 21 (ObjNum 96 snap 7 seq_num 96) dirty exists 2023-03-29T14:02:36.411 INFO:tasks.rados.rados.0.smithi017.stdout:150: left oid 1 (ObjNum 96 snap 7 seq_num 96) 2023-03-29T14:02:36.490 INFO:tasks.rados.rados.0.smithi017.stdout:150: done (1 left) 2023-03-29T14:02:36.490 INFO:tasks.rados.rados.0.smithi017.stdout:151: done (0 left) 2023-03-29T14:02:36.490 INFO:tasks.rados.rados.0.smithi017.stdout:152: write (excl) oid 40 current snap is 7 2023-03-29T14:02:36.490 INFO:tasks.rados.rados.0.smithi017.stdout:152: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2023-03-29T14:02:36.496 INFO:tasks.rados.rados.0.smithi017.stdout:152: writing smithi017107884-40 from 447086 to 1217724 tid 1 2023-03-29T14:02:36.505 INFO:tasks.rados.rados.0.smithi017.stdout:152: writing smithi017107884-40 from 1935000 to 2733016 tid 2 2023-03-29T14:02:36.510 INFO:tasks.rados.rados.0.smithi017.stdout:152: writing smithi017107884-40 from 3458512 to 3647086 tid 3 2023-03-29T14:02:36.510 INFO:tasks.rados.rados.0.smithi017.stdout:153: read oid 26 snap -1 2023-03-29T14:02:36.510 INFO:tasks.rados.rados.0.smithi017.stdout:153: expect (ObjNum 81 snap 6 seq_num 81) 2023-03-29T14:02:36.510 INFO:tasks.rados.rados.0.smithi017.stdout:154: snap_create 2023-03-29T14:02:36.518 INFO:tasks.rados.rados.0.smithi017.stdout:152: finishing write tid 1 to smithi017107884-40 2023-03-29T14:02:36.546 INFO:tasks.rados.rados.0.smithi017.stdout:153: expect (ObjNum 81 snap 6 seq_num 81) 2023-03-29T14:02:36.676 INFO:tasks.rados.rados.0.smithi017.stdout:152: finishing write tid 2 to smithi017107884-40 2023-03-29T14:02:36.676 INFO:tasks.rados.rados.0.smithi017.stdout:152: finishing write tid 3 to smithi017107884-40 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:152: finishing write tid 4 to smithi017107884-40 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:152: finishing write tid 5 to smithi017107884-40 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:152: oid 40 updating version 0 to 31 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:152: oid 40 version 31 is already newer than 31 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:152: oid 40 version 31 is already newer than 28 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:152: oid 40 version 31 is already newer than 31 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 31 (ObjNum 98 snap 7 seq_num 98) dirty exists 2023-03-29T14:02:36.677 INFO:tasks.rados.rados.0.smithi017.stdout:152: left oid 40 (ObjNum 98 snap 7 seq_num 98) 2023-03-29T14:02:36.717 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:36.717 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:36.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:36.849 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:36.846+0000 7f1277ebe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:36.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:36.853+0000 7f1277ebe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:37.061 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:37.073 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-03-29T14:02:36.099987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233114498,"num_objects":132,"num_object_clones":36,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":520,"num_read_kb":141341,"num_write":888,"num_write_kb":175298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":249952,"kb_used_data":226816,"kb_used_omap":0,"kb_used_meta":23104,"kb_avail":377237408,"statfs":{"total":386547056640,"available":386291105792,"internally_reserved":0,"allocated":232259584,"data_stored":250736836,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":23658496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":12853,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000955"},"pg_stats":[{"pgid":"3.f","version":"21'14","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.607745+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:30.607745+0000","last_peered":"2023-03-29T14:02:30.607745+0000","last_clean":"2023-03-29T14:02:30.607745+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:30.607745+0000","last_undegraded":"2023-03-29T14:02:30.607745+0000","last_fullsized":"2023-03-29T14:02:30.607745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10890804,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":11672,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"17'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.216989+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:29.216989+0000","last_peered":"2023-03-29T14:02:29.216989+0000","last_clean":"2023-03-29T14:02:29.216989+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:29.216989+0000","last_undegraded":"2023-03-29T14:02:29.216989+0000","last_fullsized":"2023-03-29T14:02:29.216989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243718,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"21'22","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:28.289167+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:28.289167+0000","last_peered":"2023-03-29T14:02:28.289167+0000","last_clean":"2023-03-29T14:02:28.289167+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:28.289167+0000","last_undegraded":"2023-03-29T14:02:28.289167+0000","last_fullsized":"2023-03-29T14:02:28.289167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15697540,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":29,"num_read_kb":7048,"num_write":50,"num_write_kb":13376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"24'23","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.354817+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:30.354817+0000","last_peered":"2023-03-29T14:02:30.354817+0000","last_clean":"2023-03-29T14:02:30.354817+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:30.354817+0000","last_undegraded":"2023-03-29T14:02:30.354817+0000","last_fullsized":"2023-03-29T14:02:30.354817+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16100908,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":23,"num_read_kb":5057,"num_write":52,"num_write_kb":13740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"19'20","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.195635+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:29.195635+0000","last_peered":"2023-03-29T14:02:29.195635+0000","last_clean":"2023-03-29T14:02:29.195635+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:29.195635+0000","last_undegraded":"2023-03-29T14:02:29.195635+0000","last_fullsized":"2023-03-29T14:02:29.195635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21882450,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":18883,"num_write":48,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:28.294157+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:28.294157+0000","last_peered":"2023-03-29T14:02:28.294157+0000","last_clean":"2023-03-29T14:02:28.294157+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:28.294157+0000","last_undegraded":"2023-03-29T14:02:28.294157+0000","last_fullsized":"2023-03-29T14:02:28.294157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"21'14","reported_seq":48,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:28.266025+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:28.266025+0000","last_peered":"2023-03-29T14:02:28.266025+0000","last_clean":"2023-03-29T14:02:28.266025+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:28.266025+0000","last_undegraded":"2023-03-29T14:02:28.266025+0000","last_fullsized":"2023-03-29T14:02:28.266025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9579858,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":14,"num_read_kb":4,"num_write":32,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":61,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.527982+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:30.527982+0000","last_peered":"2023-03-29T14:02:30.527982+0000","last_clean":"2023-03-29T14:02:30.527982+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:30.527982+0000","last_undegraded":"2023-03-29T14:02:30.527982+0000","last_fullsized":"2023-03-29T14:02:30.527982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":7452,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.205318+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:29.205318+0000","last_peered":"2023-03-29T14:02:29.205318+0000","last_clean":"2023-03-29T14:02:29.205318+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:29.205318+0000","last_undegraded":"2023-03-29T14:02:29.205318+0000","last_fullsized":"2023-03-29T14:02:29.205318+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T14:02:31.284823+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:31.284823+0000","last_peered":"2023-03-29T14:02:31.284823+0000","last_clean":"2023-03-29T14:02:31.284823+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:31.284823+0000","last_undegraded":"2023-03-29T14:02:31.284823+0000","last_fullsized":"2023-03-29T14:02:31.284823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10184910,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":13755,"num_write":16,"num_write_kb":4726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:28.278955+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:28.278955+0000","last_peered":"2023-03-29T14:02:28.278955+0000","last_clean":"2023-03-29T14:02:28.278955+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:28.278955+0000","last_undegraded":"2023-03-29T14:02:28.278955+0000","last_fullsized":"2023-03-29T14:02:28.278955+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"16'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.261139+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:29.261139+0000","last_peered":"2023-03-29T14:02:29.261139+0000","last_clean":"2023-03-29T14:02:29.261139+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:29.261139+0000","last_undegraded":"2023-03-29T14:02:29.261139+0000","last_fullsized":"2023-03-29T14:02:29.261139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714314,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":5878,"num_write":16,"num_write_kb":3954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.307665+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:30.307665+0000","last_peered":"2023-03-29T14:02:30.307665+0000","last_clean":"2023-03-29T14:02:30.307665+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:30.307665+0000","last_undegraded":"2023-03-29T14:02:30.307665+0000","last_fullsized":"2023-03-29T14:02:30.307665+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"26'19","reported_seq":69,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.620788+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:30.620788+0000","last_peered":"2023-03-29T14:02:30.620788+0000","last_clean":"2023-03-29T14:02:30.620788+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:30.620788+0000","last_undegraded":"2023-03-29T14:02:30.620788+0000","last_fullsized":"2023-03-29T14:02:30.620788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18285818,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":23599,"num_write":44,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:28.287271+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:28.287271+0000","last_peered":"2023-03-29T14:02:28.287271+0000","last_clean":"2023-03-29T14:02:28.287271+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:28.287271+0000","last_undegraded":"2023-03-29T14:02:28.287271+0000","last_fullsized":"2023-03-29T14:02:28.287271+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":246,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.232686+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:29.232686+0000","last_peered":"2023-03-29T14:02:29.232686+0000","last_clean":"2023-03-29T14:02:29.232686+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:29.232686+0000","last_undegraded":"2023-03-29T14:02:29.232686+0000","last_fullsized":"2023-03-29T14:02:29.232686+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"26'28","reported_seq":86,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.650765+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:30.650765+0000","last_peered":"2023-03-29T14:02:30.650765+0000","last_clean":"2023-03-29T14:02:30.650765+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:30.650765+0000","last_undegraded":"2023-03-29T14:02:30.650765+0000","last_fullsized":"2023-03-29T14:02:30.650765+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24100974,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":16131,"num_write":62,"num_write_kb":16994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:28.292892+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:28.292892+0000","last_peered":"2023-03-29T14:02:28.292892+0000","last_clean":"2023-03-29T14:02:28.292892+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:28.292892+0000","last_undegraded":"2023-03-29T14:02:28.292892+0000","last_fullsized":"2023-03-29T14:02:28.292892+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"26'37","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.702963+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:30.702963+0000","last_peered":"2023-03-29T14:02:30.702963+0000","last_clean":"2023-03-29T14:02:30.702963+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:30.702963+0000","last_undegraded":"2023-03-29T14:02:30.702963+0000","last_fullsized":"2023-03-29T14:02:30.702963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26319432,"num_objects":14,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":5290,"num_write":84,"num_write_kb":23918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.244113+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:29.244113+0000","last_peered":"2023-03-29T14:02:29.244113+0000","last_clean":"2023-03-29T14:02:29.244113+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:29.244113+0000","last_undegraded":"2023-03-29T14:02:29.244113+0000","last_fullsized":"2023-03-29T14:02:29.244113+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"26'20","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.733884+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:30.733884+0000","last_peered":"2023-03-29T14:02:30.733884+0000","last_clean":"2023-03-29T14:02:30.733884+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:30.733884+0000","last_undegraded":"2023-03-29T14:02:30.733884+0000","last_fullsized":"2023-03-29T14:02:30.733884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11899224,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":8642,"num_write":44,"num_write_kb":13190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'19","reported_seq":65,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.242955+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:29.242955+0000","last_peered":"2023-03-29T14:02:29.242955+0000","last_clean":"2023-03-29T14:02:29.242955+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:29.242955+0000","last_undegraded":"2023-03-29T14:02:29.242955+0000","last_fullsized":"2023-03-29T14:02:29.242955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243784,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":6072,"num_write":50,"num_write_kb":13952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:29.233952+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:29.233952+0000","last_peered":"2023-03-29T14:02:29.233952+0000","last_clean":"2023-03-29T14:02:29.233952+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:29.233952+0000","last_undegraded":"2023-03-29T14:02:29.233952+0000","last_fullsized":"2023-03-29T14:02:29.233952+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-29T14:02:28.275143+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:28.275143+0000","last_peered":"2023-03-29T14:02:28.275143+0000","last_clean":"2023-03-29T14:02:28.275143+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:28.275143+0000","last_undegraded":"2023-03-29T14:02:28.275143+0000","last_fullsized":"2023-03-29T14:02:28.275143+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'13","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T14:02:30.581165+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:30.581165+0000","last_peered":"2023-03-29T14:02:30.581165+0000","last_clean":"2023-03-29T14:02:30.581165+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:30.581165+0000","last_undegraded":"2023-03-29T14:02:30.581165+0000","last_fullsized":"2023-03-29T14:02:30.581165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13915998,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":11737,"num_write":32,"num_write_kb":8022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":231933762,"num_objects":128,"num_object_clones":36,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":378,"num_read_kb":141221,"num_write":614,"num_write_kb":171510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68096,"kb_used_data":62520,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":94303744,"statfs":{"total":96636764160,"available":96567033856,"internally_reserved":0,"allocated":64020480,"data_stored":69240079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705690,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45524,"kb_used_data":39820,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94326316,"statfs":{"total":96636764160,"available":96590147584,"internally_reserved":0,"allocated":40775680,"data_stored":42733044,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70612,"kb_used_data":64652,"kb_used_omap":0,"kb_used_meta":5952,"kb_avail":94301228,"statfs":{"total":96636764160,"available":96564457472,"internally_reserved":0,"allocated":66203648,"data_stored":72241061,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6094848},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65720,"kb_used_data":59824,"kb_used_omap":0,"kb_used_meta":5888,"kb_avail":94306120,"statfs":{"total":96636764160,"available":96569466880,"internally_reserved":0,"allocated":61259776,"data_stored":66522652,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6029312},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:37.266 INFO:tasks.rados.rados.0.smithi017.stdout:152: done (2 left) 2023-03-29T14:02:37.267 INFO:tasks.rados.rados.0.smithi017.stdout:153: done (1 left) 2023-03-29T14:02:37.267 INFO:tasks.rados.rados.0.smithi017.stdout:154: done (0 left) 2023-03-29T14:02:37.267 INFO:tasks.rados.rados.0.smithi017.stdout:155: read oid 33 snap -1 2023-03-29T14:02:37.267 INFO:tasks.rados.rados.0.smithi017.stdout:155: expect (ObjNum 32 snap 0 seq_num 32) 2023-03-29T14:02:37.267 INFO:tasks.rados.rados.0.smithi017.stdout:156: write (excl) oid 26 current snap is 8 2023-03-29T14:02:37.267 INFO:tasks.rados.rados.0.smithi017.stdout:156: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2023-03-29T14:02:37.272 INFO:tasks.rados.rados.0.smithi017.stdout:156: writing smithi017107884-26 from 463893 to 1026133 tid 1 2023-03-29T14:02:37.279 INFO:tasks.rados.rados.0.smithi017.stdout:156: writing smithi017107884-26 from 1593482 to 2251509 tid 2 2023-03-29T14:02:37.284 INFO:tasks.rados.rados.0.smithi017.stdout:156: writing smithi017107884-26 from 2685935 to 3245276 tid 3 2023-03-29T14:02:37.284 INFO:tasks.rados.rados.0.smithi017.stdout:157: delete oid 5 current snap is 8 2023-03-29T14:02:37.342 INFO:tasks.rados.rados.0.smithi017.stdout:155: expect (ObjNum 32 snap 0 seq_num 32) 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: finishing write tid 1 to smithi017107884-26 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: finishing write tid 2 to smithi017107884-26 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: finishing write tid 3 to smithi017107884-26 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: finishing write tid 4 to smithi017107884-26 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: finishing write tid 5 to smithi017107884-26 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: oid 26 updating version 0 to 33 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: oid 26 version 33 is already newer than 33 2023-03-29T14:02:37.444 INFO:tasks.rados.rados.0.smithi017.stdout:156: oid 26 version 33 is already newer than 29 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:156: oid 26 version 33 is already newer than 33 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 33 (ObjNum 99 snap 8 seq_num 99) dirty exists 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:156: left oid 26 (ObjNum 99 snap 8 seq_num 99) 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:155: done (2 left) 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:156: done (1 left) 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:157: done (0 left) 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:158: read oid 45 snap 7 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:158: expect (ObjNum 61 snap 2 seq_num 61) 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:159: read oid 31 snap -1 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:159: expect (ObjNum 30 snap 0 seq_num 30) 2023-03-29T14:02:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:160: snap_create 2023-03-29T14:02:37.454 INFO:tasks.rados.rados.0.smithi017.stdout:158: expect (ObjNum 61 snap 2 seq_num 61) 2023-03-29T14:02:37.614 INFO:tasks.rados.rados.0.smithi017.stdout:159: expect (ObjNum 30 snap 0 seq_num 30) 2023-03-29T14:02:38.032 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:38.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:38.163+0000 7f8d6c21c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:38.168+0000 7f8d6c21c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:38.270 INFO:tasks.rados.rados.0.smithi017.stdout:158: done (2 left) 2023-03-29T14:02:38.270 INFO:tasks.rados.rados.0.smithi017.stdout:159: done (1 left) 2023-03-29T14:02:38.270 INFO:tasks.rados.rados.0.smithi017.stdout:160: done (0 left) 2023-03-29T14:02:38.270 INFO:tasks.rados.rados.0.smithi017.stdout:161: snap_remove snap 8 2023-03-29T14:02:39.271 INFO:tasks.rados.rados.0.smithi017.stdout:161: done (0 left) 2023-03-29T14:02:39.271 INFO:tasks.rados.rados.0.smithi017.stdout:162: write (excl) oid 26 current snap is 9 2023-03-29T14:02:39.271 INFO:tasks.rados.rados.0.smithi017.stdout:162: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2023-03-29T14:02:39.277 INFO:tasks.rados.rados.0.smithi017.stdout:162: writing smithi017107884-26 from 497507 to 1210245 tid 1 2023-03-29T14:02:39.284 INFO:tasks.rados.rados.0.smithi017.stdout:162: writing smithi017107884-26 from 1710446 to 2372142 tid 2 2023-03-29T14:02:39.291 INFO:tasks.rados.rados.0.smithi017.stdout:162: writing smithi017107884-26 from 3140781 to 3697507 tid 3 2023-03-29T14:02:39.291 INFO:tasks.rados.rados.0.smithi017.stdout:163: snap_remove snap 7 2023-03-29T14:02:40.238 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:40.277 INFO:tasks.rados.rados.0.smithi017.stdout:163: done (1 left) 2023-03-29T14:02:40.277 INFO:tasks.rados.rados.0.smithi017.stdout:164: read oid 7 snap -1 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:164: expect (ObjNum 67 snap 3 seq_num 67) 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:165: delete oid 12 current snap is 9 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: finishing write tid 1 to smithi017107884-26 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: finishing write tid 2 to smithi017107884-26 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: finishing write tid 3 to smithi017107884-26 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: finishing write tid 4 to smithi017107884-26 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: finishing write tid 5 to smithi017107884-26 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: oid 26 updating version 0 to 34 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: oid 26 updating version 34 to 37 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: oid 26 version 37 is already newer than 37 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: oid 26 version 37 is already newer than 37 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 37 (ObjNum 101 snap 9 seq_num 101) dirty exists 2023-03-29T14:02:40.278 INFO:tasks.rados.rados.0.smithi017.stdout:162: left oid 26 (ObjNum 101 snap 9 seq_num 101) 2023-03-29T14:02:40.354 INFO:tasks.rados.rados.0.smithi017.stdout:164: expect (ObjNum 67 snap 3 seq_num 67) 2023-03-29T14:02:40.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:40.386+0000 7fb7aa82e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:40.390 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:40.391+0000 7fb7aa82e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:40.519 INFO:tasks.rados.rados.0.smithi017.stdout:162: done (2 left) 2023-03-29T14:02:40.519 INFO:tasks.rados.rados.0.smithi017.stdout:164: done (1 left) 2023-03-29T14:02:40.519 INFO:tasks.rados.rados.0.smithi017.stdout:165: done (0 left) 2023-03-29T14:02:40.519 INFO:tasks.rados.rados.0.smithi017.stdout:166: delete oid 48 current snap is 9 2023-03-29T14:02:40.564 INFO:tasks.rados.rados.0.smithi017.stdout:166: done (0 left) 2023-03-29T14:02:40.564 INFO:tasks.rados.rados.0.smithi017.stdout:167: delete oid 11 current snap is 9 2023-03-29T14:02:40.660 INFO:tasks.rados.rados.0.smithi017.stdout:167: done (0 left) 2023-03-29T14:02:40.661 INFO:tasks.rados.rados.0.smithi017.stdout:168: write (excl) oid 44 current snap is 9 2023-03-29T14:02:40.661 INFO:tasks.rados.rados.0.smithi017.stdout:168: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2023-03-29T14:02:40.667 INFO:tasks.rados.rados.0.smithi017.stdout:168: writing smithi017107884-44 from 564735 to 1178469 tid 1 2023-03-29T14:02:40.674 INFO:tasks.rados.rados.0.smithi017.stdout:168: writing smithi017107884-44 from 1828021 to 2497055 tid 2 2023-03-29T14:02:40.681 INFO:tasks.rados.rados.0.smithi017.stdout:168: writing smithi017107884-44 from 3134120 to 3764735 tid 3 2023-03-29T14:02:40.681 INFO:tasks.rados.rados.0.smithi017.stdout:169: write oid 2 current snap is 9 2023-03-29T14:02:40.682 INFO:tasks.rados.rados.0.smithi017.stdout:169: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2023-03-29T14:02:40.688 INFO:tasks.rados.rados.0.smithi017.stdout:169: writing smithi017107884-2 from 581542 to 1270525 tid 1 2023-03-29T14:02:40.694 INFO:tasks.rados.rados.0.smithi017.stdout:169: writing smithi017107884-2 from 1886503 to 2299195 tid 2 2023-03-29T14:02:40.700 INFO:tasks.rados.rados.0.smithi017.stdout:169: writing smithi017107884-2 from 2761543 to 3432494 tid 3 2023-03-29T14:02:40.700 INFO:tasks.rados.rados.0.smithi017.stdout:170: read oid 21 snap -1 2023-03-29T14:02:40.701 INFO:tasks.rados.rados.0.smithi017.stdout:170: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:02:40.701 INFO:tasks.rados.rados.0.smithi017.stdout:171: write oid 17 current snap is 9 2023-03-29T14:02:40.701 INFO:tasks.rados.rados.0.smithi017.stdout:171: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2023-03-29T14:02:40.706 INFO:tasks.rados.rados.0.smithi017.stdout:171: writing smithi017107884-17 from 598349 to 1078934 tid 1 2023-03-29T14:02:40.713 INFO:tasks.rados.rados.0.smithi017.stdout:171: writing smithi017107884-17 from 1544985 to 2217688 tid 2 2023-03-29T14:02:40.720 INFO:tasks.rados.rados.0.smithi017.stdout:171: writing smithi017107884-17 from 2905319 to 3387542 tid 3 2023-03-29T14:02:40.720 INFO:tasks.rados.rados.0.smithi017.stdout:172: write (excl) oid 15 current snap is 9 2023-03-29T14:02:40.720 INFO:tasks.rados.rados.0.smithi017.stdout:168: finishing write tid 1 to smithi017107884-44 2023-03-29T14:02:40.720 INFO:tasks.rados.rados.0.smithi017.stdout:172: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2023-03-29T14:02:40.725 INFO:tasks.rados.rados.0.smithi017.stdout:172: writing smithi017107884-15 from 615156 to 1170990 tid 1 2023-03-29T14:02:40.730 INFO:tasks.rados.rados.0.smithi017.stdout:172: writing smithi017107884-15 from 1603467 to 2019828 tid 2 2023-03-29T14:02:40.739 INFO:tasks.rados.rados.0.smithi017.stdout:172: writing smithi017107884-15 from 2816389 to 3509884 tid 3 2023-03-29T14:02:40.739 INFO:tasks.rados.rados.0.smithi017.stdout:173: read oid 9 snap -1 2023-03-29T14:02:40.739 INFO:tasks.rados.rados.0.smithi017.stdout:173: expect (ObjNum 8 snap 0 seq_num 8) 2023-03-29T14:02:40.739 INFO:tasks.rados.rados.0.smithi017.stdout:174: write (excl) oid 39 current snap is 9 2023-03-29T14:02:40.739 INFO:tasks.rados.rados.0.smithi017.stdout:174: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2023-03-29T14:02:40.746 INFO:tasks.rados.rados.0.smithi017.stdout:174: writing smithi017107884-39 from 631963 to 1263046 tid 1 2023-03-29T14:02:40.753 INFO:tasks.rados.rados.0.smithi017.stdout:174: writing smithi017107884-39 from 2061949 to 2621968 tid 2 2023-03-29T14:02:40.760 INFO:tasks.rados.rados.0.smithi017.stdout:174: writing smithi017107884-39 from 3243812 to 3748579 tid 3 2023-03-29T14:02:40.761 INFO:tasks.rados.rados.0.smithi017.stdout:175: read oid 3 snap -1 2023-03-29T14:02:40.761 INFO:tasks.rados.rados.0.smithi017.stdout:175: expect (ObjNum 91 snap 7 seq_num 91) 2023-03-29T14:02:40.761 INFO:tasks.rados.rados.0.smithi017.stdout:176: write oid 35 current snap is 9 2023-03-29T14:02:40.761 INFO:tasks.rados.rados.0.smithi017.stdout:176: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2023-03-29T14:02:40.767 INFO:tasks.rados.rados.0.smithi017.stdout:176: writing smithi017107884-35 from 648770 to 1355102 tid 1 2023-03-29T14:02:40.777 INFO:tasks.rados.rados.0.smithi017.stdout:176: writing smithi017107884-35 from 2120431 to 2540461 tid 2 2023-03-29T14:02:40.781 INFO:tasks.rados.rados.0.smithi017.stdout:176: writing smithi017107884-35 from 3271235 to 3703627 tid 3 2023-03-29T14:02:40.781 INFO:tasks.rados.rados.0.smithi017.stdout:177: snap_create 2023-03-29T14:02:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:169: finishing write tid 1 to smithi017107884-2 2023-03-29T14:02:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:170: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:168: finishing write tid 2 to smithi017107884-44 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:168: finishing write tid 3 to smithi017107884-44 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:171: finishing write tid 1 to smithi017107884-17 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:169: finishing write tid 2 to smithi017107884-2 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:169: finishing write tid 3 to smithi017107884-2 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:169: finishing write tid 4 to smithi017107884-2 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:169: finishing write tid 5 to smithi017107884-2 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:169: oid 2 updating version 0 to 51 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:169: oid 2 version 51 is already newer than 51 2023-03-29T14:02:40.882 INFO:tasks.rados.rados.0.smithi017.stdout:169: oid 2 version 51 is already newer than 51 2023-03-29T14:02:40.883 INFO:tasks.rados.rados.0.smithi017.stdout:169: oid 2 version 51 is already newer than 51 2023-03-29T14:02:40.883 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 51 (ObjNum 106 snap 9 seq_num 106) dirty exists 2023-03-29T14:02:40.883 INFO:tasks.rados.rados.0.smithi017.stdout:169: left oid 2 (ObjNum 106 snap 9 seq_num 106) 2023-03-29T14:02:40.883 INFO:tasks.rados.rados.0.smithi017.stdout:168: finishing write tid 4 to smithi017107884-44 2023-03-29T14:02:40.883 INFO:tasks.rados.rados.0.smithi017.stdout:173: expect (ObjNum 8 snap 0 seq_num 8) 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:171: finishing write tid 2 to smithi017107884-17 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:168: finishing write tid 5 to smithi017107884-44 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:168: oid 44 updating version 0 to 25 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:168: oid 44 updating version 25 to 28 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:168: oid 44 version 28 is already newer than 28 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:168: oid 44 version 28 is already newer than 28 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 28 (ObjNum 105 snap 9 seq_num 105) dirty exists 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:168: left oid 44 (ObjNum 105 snap 9 seq_num 105) 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:172: finishing write tid 1 to smithi017107884-15 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:176: finishing write tid 1 to smithi017107884-35 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:171: finishing write tid 3 to smithi017107884-17 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:171: finishing write tid 4 to smithi017107884-17 2023-03-29T14:02:40.975 INFO:tasks.rados.rados.0.smithi017.stdout:171: finishing write tid 5 to smithi017107884-17 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:171: oid 17 updating version 0 to 32 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:171: oid 17 version 32 is already newer than 32 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:171: oid 17 version 32 is already newer than 32 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:171: oid 17 version 32 is already newer than 32 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 32 (ObjNum 107 snap 9 seq_num 107) dirty exists 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:171: left oid 17 (ObjNum 107 snap 9 seq_num 107) 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: finishing write tid 2 to smithi017107884-15 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: finishing write tid 3 to smithi017107884-15 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: finishing write tid 4 to smithi017107884-15 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: finishing write tid 5 to smithi017107884-15 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: oid 15 updating version 0 to 55 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: oid 15 version 55 is already newer than 55 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: oid 15 version 55 is already newer than 55 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: oid 15 version 55 is already newer than 52 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 55 (ObjNum 108 snap 9 seq_num 108) dirty exists 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:172: left oid 15 (ObjNum 108 snap 9 seq_num 108) 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:176: finishing write tid 2 to smithi017107884-35 2023-03-29T14:02:40.976 INFO:tasks.rados.rados.0.smithi017.stdout:176: finishing write tid 3 to smithi017107884-35 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:176: finishing write tid 4 to smithi017107884-35 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:176: finishing write tid 5 to smithi017107884-35 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:176: oid 35 updating version 0 to 32 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:176: oid 35 version 32 is already newer than 32 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:176: oid 35 version 32 is already newer than 32 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:176: oid 35 version 32 is already newer than 32 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 32 (ObjNum 110 snap 9 seq_num 110) dirty exists 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:176: left oid 35 (ObjNum 110 snap 9 seq_num 110) 2023-03-29T14:02:40.977 INFO:tasks.rados.rados.0.smithi017.stdout:175: expect (ObjNum 91 snap 7 seq_num 91) 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: finishing write tid 1 to smithi017107884-39 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: finishing write tid 2 to smithi017107884-39 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: finishing write tid 3 to smithi017107884-39 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: finishing write tid 4 to smithi017107884-39 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: finishing write tid 5 to smithi017107884-39 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: oid 39 updating version 0 to 36 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: oid 39 version 36 is already newer than 36 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: oid 39 version 36 is already newer than 33 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: oid 39 version 36 is already newer than 36 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 36 (ObjNum 109 snap 9 seq_num 109) dirty exists 2023-03-29T14:02:41.150 INFO:tasks.rados.rados.0.smithi017.stdout:174: left oid 39 (ObjNum 109 snap 9 seq_num 109) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:168: done (9 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:169: done (8 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:170: done (7 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:171: done (6 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:172: done (5 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:173: done (4 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:174: done (3 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:175: done (2 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:176: done (1 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:177: done (0 left) 2023-03-29T14:02:41.278 INFO:tasks.rados.rados.0.smithi017.stdout:178: delete oid 23 current snap is 10 2023-03-29T14:02:41.350 INFO:tasks.rados.rados.0.smithi017.stdout:178: done (0 left) 2023-03-29T14:02:41.351 INFO:tasks.rados.rados.0.smithi017.stdout:179: delete oid 36 current snap is 10 2023-03-29T14:02:41.419 INFO:tasks.rados.rados.0.smithi017.stdout:179: done (0 left) 2023-03-29T14:02:41.419 INFO:tasks.rados.rados.0.smithi017.stdout:180: write oid 41 current snap is 10 2023-03-29T14:02:41.419 INFO:tasks.rados.rados.0.smithi017.stdout:180: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2023-03-29T14:02:41.425 INFO:tasks.rados.rados.0.smithi017.stdout:180: writing smithi017107884-41 from 699191 to 1231270 tid 1 2023-03-29T14:02:41.432 INFO:tasks.rados.rados.0.smithi017.stdout:180: writing smithi017107884-41 from 1779524 to 2346881 tid 2 2023-03-29T14:02:41.438 INFO:tasks.rados.rados.0.smithi017.stdout:180: writing smithi017107884-41 from 2837151 to 3503359 tid 3 2023-03-29T14:02:41.438 INFO:tasks.rados.rados.0.smithi017.stdout:181: write oid 20 current snap is 10 2023-03-29T14:02:41.438 INFO:tasks.rados.rados.0.smithi017.stdout:181: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2023-03-29T14:02:41.445 INFO:tasks.rados.rados.0.smithi017.stdout:181: writing smithi017107884-20 from 715998 to 1323326 tid 1 2023-03-29T14:02:41.451 INFO:tasks.rados.rados.0.smithi017.stdout:181: writing smithi017107884-20 from 1838006 to 2265374 tid 2 2023-03-29T14:02:41.456 INFO:tasks.rados.rados.0.smithi017.stdout:181: writing smithi017107884-20 from 2864574 to 3342054 tid 3 2023-03-29T14:02:41.460 INFO:tasks.rados.rados.0.smithi017.stdout:181: writing smithi017107884-20 from 3863835 to 3915998 tid 4 2023-03-29T14:02:41.460 INFO:tasks.rados.rados.0.smithi017.stdout:182: snap_remove snap 9 2023-03-29T14:02:42.077 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:42.078 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:42.078 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:42.210 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:42.207+0000 7f275558f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:42.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:42.214+0000 7f275558f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:42.283 INFO:tasks.rados.rados.0.smithi017.stdout:182: done (2 left) 2023-03-29T14:02:42.283 INFO:tasks.rados.rados.0.smithi017.stdout:183: delete oid 30 current snap is 10 2023-03-29T14:02:42.283 INFO:tasks.rados.rados.0.smithi017.stdout:180: finishing write tid 1 to smithi017107884-41 2023-03-29T14:02:42.283 INFO:tasks.rados.rados.0.smithi017.stdout:180: finishing write tid 2 to smithi017107884-41 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: finishing write tid 3 to smithi017107884-41 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: finishing write tid 4 to smithi017107884-41 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: finishing write tid 5 to smithi017107884-41 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: oid 41 updating version 0 to 31 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: oid 41 version 31 is already newer than 31 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: oid 41 version 31 is already newer than 29 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: oid 41 version 31 is already newer than 31 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 31 (ObjNum 113 snap 10 seq_num 113) dirty exists 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:180: left oid 41 (ObjNum 113 snap 10 seq_num 113) 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: finishing write tid 1 to smithi017107884-20 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: finishing write tid 2 to smithi017107884-20 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: finishing write tid 3 to smithi017107884-20 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: finishing write tid 4 to smithi017107884-20 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: finishing write tid 5 to smithi017107884-20 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: finishing write tid 6 to smithi017107884-20 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: oid 20 updating version 0 to 30 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: oid 20 version 30 is already newer than 30 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: oid 20 version 30 is already newer than 30 2023-03-29T14:02:42.284 INFO:tasks.rados.rados.0.smithi017.stdout:181: oid 20 version 30 is already newer than 30 2023-03-29T14:02:42.285 INFO:tasks.rados.rados.0.smithi017.stdout:181: oid 20 version 30 is already newer than 30 2023-03-29T14:02:42.285 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 30 (ObjNum 114 snap 10 seq_num 114) dirty exists 2023-03-29T14:02:42.285 INFO:tasks.rados.rados.0.smithi017.stdout:181: left oid 20 (ObjNum 114 snap 10 seq_num 114) 2023-03-29T14:02:42.423 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:42.425 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:42.437 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2023-03-29T14:02:42.282454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":260476360,"num_objects":134,"num_object_clones":40,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":687,"num_read_kb":243232,"num_write":1108,"num_write_kb":245714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":455,"ondisk_log_size":455,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":310536,"kb_used_data":272392,"kb_used_omap":0,"kb_used_meta":24576,"kb_avail":377176824,"statfs":{"total":386547056640,"available":386229067776,"internally_reserved":0,"allocated":278929408,"data_stored":292245594,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":25165824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2184712,"num_objects":0,"num_object_clones":4,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":82,"num_read_kb":49363,"num_write":100,"num_write_kb":33886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013540"},"pg_stats":[{"pgid":"3.f","version":"21'14","reported_seq":66,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.326555+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:40.326555+0000","last_peered":"2023-03-29T14:02:40.326555+0000","last_clean":"2023-03-29T14:02:40.326555+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:40.326555+0000","last_undegraded":"2023-03-29T14:02:40.326555+0000","last_fullsized":"2023-03-29T14:02:40.326555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10890804,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":24329,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"17'3","reported_seq":30,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.307147+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:37.307147+0000","last_peered":"2023-03-29T14:02:37.307147+0000","last_clean":"2023-03-29T14:02:37.307147+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:37.307147+0000","last_undegraded":"2023-03-29T14:02:37.307147+0000","last_fullsized":"2023-03-29T14:02:37.307147+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243718,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"35'36","reported_seq":103,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:41.002033+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:41.002033+0000","last_peered":"2023-03-29T14:02:41.002033+0000","last_clean":"2023-03-29T14:02:41.002033+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:41.002033+0000","last_undegraded":"2023-03-29T14:02:41.002033+0000","last_fullsized":"2023-03-29T14:02:41.002033+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143008,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":34,"num_read_kb":7051,"num_write":80,"num_write_kb":22982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"31'31","reported_seq":99,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.814073+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:40.814073+0000","last_peered":"2023-03-29T14:02:40.814073+0000","last_clean":"2023-03-29T14:02:40.814073+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:40.814073+0000","last_undegraded":"2023-03-29T14:02:40.814073+0000","last_fullsized":"2023-03-29T14:02:40.814073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24336272,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":15161,"num_write":72,"num_write_kb":20306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"31'24","reported_seq":89,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.454078+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:37.454078+0000","last_peered":"2023-03-29T14:02:37.454078+0000","last_clean":"2023-03-29T14:02:37.454078+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:37.454078+0000","last_undegraded":"2023-03-29T14:02:37.454078+0000","last_fullsized":"2023-03-29T14:02:37.454078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23932904,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":28562,"num_write":58,"num_write_kb":17796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:39.344365+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:39.344365+0000","last_peered":"2023-03-29T14:02:39.344365+0000","last_clean":"2023-03-29T14:02:39.344365+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:39.344365+0000","last_undegraded":"2023-03-29T14:02:39.344365+0000","last_fullsized":"2023-03-29T14:02:39.344365+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"31'18","reported_seq":66,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:39.342145+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:39.342145+0000","last_peered":"2023-03-29T14:02:39.342145+0000","last_clean":"2023-03-29T14:02:39.342145+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:39.342145+0000","last_undegraded":"2023-03-29T14:02:39.342145+0000","last_fullsized":"2023-03-29T14:02:39.342145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16672346,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":6098,"num_write":42,"num_write_kb":11466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":70,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.345666+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:40.345666+0000","last_peered":"2023-03-29T14:02:40.345666+0000","last_clean":"2023-03-29T14:02:40.345666+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:40.345666+0000","last_undegraded":"2023-03-29T14:02:40.345666+0000","last_fullsized":"2023-03-29T14:02:40.345666+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15576,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.298130+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:37.298130+0000","last_peered":"2023-03-29T14:02:37.298130+0000","last_clean":"2023-03-29T14:02:37.298130+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:37.298130+0000","last_undegraded":"2023-03-29T14:02:37.298130+0000","last_fullsized":"2023-03-29T14:02:37.298130+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":44,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.318681+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:40.318681+0000","last_peered":"2023-03-29T14:02:40.318681+0000","last_clean":"2023-03-29T14:02:40.318681+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:40.318681+0000","last_undegraded":"2023-03-29T14:02:40.318681+0000","last_fullsized":"2023-03-29T14:02:40.318681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10184910,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":13755,"num_write":16,"num_write_kb":4726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:39.338828+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:39.338828+0000","last_peered":"2023-03-29T14:02:39.338828+0000","last_clean":"2023-03-29T14:02:39.338828+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:39.338828+0000","last_undegraded":"2023-03-29T14:02:39.338828+0000","last_fullsized":"2023-03-29T14:02:39.338828+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":51,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.302789+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:37.302789+0000","last_peered":"2023-03-29T14:02:37.302789+0000","last_clean":"2023-03-29T14:02:37.302789+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:37.302789+0000","last_undegraded":"2023-03-29T14:02:37.302789+0000","last_fullsized":"2023-03-29T14:02:37.302789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":5879,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.331646+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:40.331646+0000","last_peered":"2023-03-29T14:02:40.331646+0000","last_clean":"2023-03-29T14:02:40.331646+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:40.331646+0000","last_undegraded":"2023-03-29T14:02:40.331646+0000","last_fullsized":"2023-03-29T14:02:40.331646+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"29'27","reported_seq":92,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.318368+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:37.318368+0000","last_peered":"2023-03-29T14:02:37.318368+0000","last_clean":"2023-03-29T14:02:37.318368+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:37.318368+0000","last_undegraded":"2023-03-29T14:02:37.318368+0000","last_fullsized":"2023-03-29T14:02:37.318368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369886,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":23601,"num_write":64,"num_write_kb":19822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:39.315768+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:39.315768+0000","last_peered":"2023-03-29T14:02:39.315768+0000","last_clean":"2023-03-29T14:02:39.315768+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:39.315768+0000","last_undegraded":"2023-03-29T14:02:39.315768+0000","last_fullsized":"2023-03-29T14:02:39.315768+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":251,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.321751+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:37.321751+0000","last_peered":"2023-03-29T14:02:37.321751+0000","last_clean":"2023-03-29T14:02:37.321751+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:37.321751+0000","last_undegraded":"2023-03-29T14:02:37.321751+0000","last_fullsized":"2023-03-29T14:02:37.321751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"34'37","reported_seq":116,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.767021+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:40.767021+0000","last_peered":"2023-03-29T14:02:40.767021+0000","last_clean":"2023-03-29T14:02:40.767021+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:40.767021+0000","last_undegraded":"2023-03-29T14:02:40.767021+0000","last_fullsized":"2023-03-29T14:02:40.767021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24773254,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":28232,"num_write":82,"num_write_kb":24250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:39.347055+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:39.347055+0000","last_peered":"2023-03-29T14:02:39.347055+0000","last_clean":"2023-03-29T14:02:39.347055+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:39.347055+0000","last_undegraded":"2023-03-29T14:02:39.347055+0000","last_fullsized":"2023-03-29T14:02:39.347055+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"35'55","reported_seq":157,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.916242+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:40.916242+0000","last_peered":"2023-03-29T14:02:40.916242+0000","last_clean":"2023-03-29T14:02:40.916242+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:40.916242+0000","last_undegraded":"2023-03-29T14:02:40.916242+0000","last_fullsized":"2023-03-29T14:02:40.916242+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27294304,"num_objects":14,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":68,"num_read_kb":28666,"num_write":124,"num_write_kb":36494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.280091+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:37.280091+0000","last_peered":"2023-03-29T14:02:37.280091+0000","last_clean":"2023-03-29T14:02:37.280091+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:37.280091+0000","last_undegraded":"2023-03-29T14:02:37.280091+0000","last_fullsized":"2023-03-29T14:02:37.280091+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":98,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.847031+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:40.847031+0000","last_peered":"2023-03-29T14:02:40.847031+0000","last_clean":"2023-03-29T14:02:40.847031+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:40.847031+0000","last_undegraded":"2023-03-29T14:02:40.847031+0000","last_fullsized":"2023-03-29T14:02:40.847031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":15525,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"32'25","reported_seq":86,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:40.350772+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:40.350772+0000","last_peered":"2023-03-29T14:02:40.350772+0000","last_clean":"2023-03-29T14:02:40.350772+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:40.350772+0000","last_undegraded":"2023-03-29T14:02:40.350772+0000","last_fullsized":"2023-03-29T14:02:40.350772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18016906,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":35,"num_read_kb":11148,"num_write":60,"num_write_kb":17014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.313426+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:37.313426+0000","last_peered":"2023-03-29T14:02:37.313426+0000","last_clean":"2023-03-29T14:02:37.313426+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:37.313426+0000","last_undegraded":"2023-03-29T14:02:37.313426+0000","last_fullsized":"2023-03-29T14:02:37.313426+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-03-29T14:02:39.325817+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:39.325817+0000","last_peered":"2023-03-29T14:02:39.325817+0000","last_clean":"2023-03-29T14:02:39.325817+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:39.325817+0000","last_undegraded":"2023-03-29T14:02:39.325817+0000","last_fullsized":"2023-03-29T14:02:39.325817+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'22","reported_seq":87,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-03-29T14:02:37.341352+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:37.341352+0000","last_peered":"2023-03-29T14:02:37.341352+0000","last_clean":"2023-03-29T14:02:37.341352+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:37.341352+0000","last_undegraded":"2023-03-29T14:02:37.341352+0000","last_fullsized":"2023-03-29T14:02:37.341352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16537890,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":19528,"num_write":52,"num_write_kb":14408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":259295624,"num_objects":130,"num_object_clones":40,"num_object_copies":390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":545,"num_read_kb":243112,"num_write":834,"num_write_kb":241926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":358,"ondisk_log_size":358,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":85176,"kb_used_data":74768,"kb_used_omap":0,"kb_used_meta":5888,"kb_avail":94286664,"statfs":{"total":96636764160,"available":96549543936,"internally_reserved":0,"allocated":76562432,"data_stored":80413824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6029312},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55300,"kb_used_data":47784,"kb_used_omap":0,"kb_used_meta":6016,"kb_avail":94316540,"statfs":{"total":96636764160,"available":96580136960,"internally_reserved":0,"allocated":48930816,"data_stored":50165974,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6160384},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86652,"kb_used_data":75732,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94285188,"statfs":{"total":96636764160,"available":96548032512,"internally_reserved":0,"allocated":77549568,"data_stored":81869436,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83408,"kb_used_data":74108,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":94288432,"statfs":{"total":96636764160,"available":96551354368,"internally_reserved":0,"allocated":75886592,"data_stored":79796360,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:42.441 INFO:tasks.rados.rados.0.smithi017.stdout:180: done (2 left) 2023-03-29T14:02:42.441 INFO:tasks.rados.rados.0.smithi017.stdout:181: done (1 left) 2023-03-29T14:02:42.442 INFO:tasks.rados.rados.0.smithi017.stdout:183: done (0 left) 2023-03-29T14:02:42.442 INFO:tasks.rados.rados.0.smithi017.stdout:184: write (excl) oid 45 current snap is 10 2023-03-29T14:02:42.442 INFO:tasks.rados.rados.0.smithi017.stdout:184: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2023-03-29T14:02:42.445 INFO:tasks.rados.rados.0.smithi017.stdout:184: writing smithi017107884-45 from 749612 to 1223791 tid 1 2023-03-29T14:02:42.452 INFO:tasks.rados.rados.0.smithi017.stdout:184: writing smithi017107884-45 from 1954970 to 2386007 tid 2 2023-03-29T14:02:42.458 INFO:tasks.rados.rados.0.smithi017.stdout:184: writing smithi017107884-45 from 2919420 to 3535797 tid 3 2023-03-29T14:02:42.458 INFO:tasks.rados.rados.0.smithi017.stdout:185: write (excl) oid 4 current snap is 10 2023-03-29T14:02:42.458 INFO:tasks.rados.rados.0.smithi017.stdout:185: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2023-03-29T14:02:42.465 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:42.466 INFO:tasks.rados.rados.0.smithi017.stdout:185: writing smithi017107884-4 from 766419 to 1315847 tid 1 2023-03-29T14:02:42.472 INFO:tasks.rados.rados.0.smithi017.stdout:185: writing smithi017107884-4 from 2013452 to 2588147 tid 2 2023-03-29T14:02:42.479 INFO:tasks.rados.rados.0.smithi017.stdout:185: writing smithi017107884-4 from 3346843 to 3774492 tid 3 2023-03-29T14:02:42.480 INFO:tasks.rados.rados.0.smithi017.stdout:186: delete oid 30 current snap is 10 2023-03-29T14:02:42.493 INFO:tasks.rados.rados.0.smithi017.stdout:184: finishing write tid 1 to smithi017107884-45 2023-03-29T14:02:42.514 INFO:tasks.rados.rados.0.smithi017.stdout:185: finishing write tid 1 to smithi017107884-4 2023-03-29T14:02:42.524 INFO:tasks.rados.rados.0.smithi017.stdout:184: finishing write tid 2 to smithi017107884-45 2023-03-29T14:02:42.526 INFO:tasks.rados.rados.0.smithi017.stdout:184: finishing write tid 3 to smithi017107884-45 2023-03-29T14:02:42.526 INFO:tasks.rados.rados.0.smithi017.stdout:186: done (2 left) 2023-03-29T14:02:42.526 INFO:tasks.rados.rados.0.smithi017.stdout:187: write (excl) oid 3 current snap is 10 2023-03-29T14:02:42.526 INFO:tasks.rados.rados.0.smithi017.stdout:187: seq_num 119 ranges {400033=699926,1614063=385970} 2023-03-29T14:02:42.532 INFO:tasks.rados.rados.0.smithi017.stdout:187: writing smithi017107884-3 from 400033 to 1099959 tid 1 2023-03-29T14:02:42.537 INFO:tasks.rados.rados.0.smithi017.stdout:187: writing smithi017107884-3 from 1614063 to 2000033 tid 2 2023-03-29T14:02:42.537 INFO:tasks.rados.rados.0.smithi017.stdout:188: write oid 26 current snap is 10 2023-03-29T14:02:42.537 INFO:tasks.rados.rados.0.smithi017.stdout:188: seq_num 120 ranges {416840=775175,1672545=344295} 2023-03-29T14:02:42.544 INFO:tasks.rados.rados.0.smithi017.stdout:188: writing smithi017107884-26 from 416840 to 1192015 tid 1 2023-03-29T14:02:42.548 INFO:tasks.rados.rados.0.smithi017.stdout:188: writing smithi017107884-26 from 1672545 to 2016840 tid 2 2023-03-29T14:02:42.549 INFO:tasks.rados.rados.0.smithi017.stdout:189: snap_create 2023-03-29T14:02:42.562 INFO:tasks.rados.rados.0.smithi017.stdout:184: finishing write tid 4 to smithi017107884-45 2023-03-29T14:02:42.567 INFO:tasks.rados.rados.0.smithi017.stdout:184: finishing write tid 5 to smithi017107884-45 2023-03-29T14:02:42.567 INFO:tasks.rados.rados.0.smithi017.stdout:184: oid 45 updating version 0 to 34 2023-03-29T14:02:42.567 INFO:tasks.rados.rados.0.smithi017.stdout:184: oid 45 version 34 is already newer than 34 2023-03-29T14:02:42.567 INFO:tasks.rados.rados.0.smithi017.stdout:184: oid 45 version 34 is already newer than 34 2023-03-29T14:02:42.567 INFO:tasks.rados.rados.0.smithi017.stdout:184: oid 45 version 34 is already newer than 31 2023-03-29T14:02:42.567 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 34 (ObjNum 116 snap 10 seq_num 116) dirty exists 2023-03-29T14:02:42.567 INFO:tasks.rados.rados.0.smithi017.stdout:184: left oid 45 (ObjNum 116 snap 10 seq_num 116) 2023-03-29T14:02:42.587 INFO:tasks.rados.rados.0.smithi017.stdout:185: finishing write tid 2 to smithi017107884-4 2023-03-29T14:02:42.603 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:42.602+0000 7f7f960cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:185: finishing write tid 3 to smithi017107884-4 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:188: finishing write tid 1 to smithi017107884-26 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:185: finishing write tid 4 to smithi017107884-4 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:188: finishing write tid 2 to smithi017107884-26 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:185: finishing write tid 5 to smithi017107884-4 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:185: oid 4 updating version 0 to 40 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:185: oid 4 version 40 is already newer than 37 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:185: oid 4 version 40 is already newer than 40 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:185: oid 4 version 40 is already newer than 40 2023-03-29T14:02:42.604 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 40 (ObjNum 117 snap 10 seq_num 117) dirty exists 2023-03-29T14:02:42.605 INFO:tasks.rados.rados.0.smithi017.stdout:185: left oid 4 (ObjNum 117 snap 10 seq_num 117) 2023-03-29T14:02:42.605 INFO:tasks.rados.rados.0.smithi017.stdout:188: finishing write tid 3 to smithi017107884-26 2023-03-29T14:02:42.606 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:42.606+0000 7f7f960cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:42.645 INFO:tasks.rados.rados.0.smithi017.stdout:188: finishing write tid 4 to smithi017107884-26 2023-03-29T14:02:42.645 INFO:tasks.rados.rados.0.smithi017.stdout:188: oid 26 updating version 0 to 40 2023-03-29T14:02:42.645 INFO:tasks.rados.rados.0.smithi017.stdout:188: oid 26 version 40 is already newer than 40 2023-03-29T14:02:42.645 INFO:tasks.rados.rados.0.smithi017.stdout:188: oid 26 version 40 is already newer than 40 2023-03-29T14:02:42.645 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 40 (ObjNum 120 snap 10 seq_num 120) dirty exists 2023-03-29T14:02:42.645 INFO:tasks.rados.rados.0.smithi017.stdout:188: left oid 26 (ObjNum 120 snap 10 seq_num 120) 2023-03-29T14:02:42.646 INFO:tasks.rados.rados.0.smithi017.stdout:187: finishing write tid 1 to smithi017107884-3 2023-03-29T14:02:42.699 INFO:tasks.rados.rados.0.smithi017.stdout:187: finishing write tid 2 to smithi017107884-3 2023-03-29T14:02:42.700 INFO:tasks.rados.rados.0.smithi017.stdout:187: finishing write tid 3 to smithi017107884-3 2023-03-29T14:02:42.701 INFO:tasks.rados.rados.0.smithi017.stdout:187: finishing write tid 4 to smithi017107884-3 2023-03-29T14:02:42.701 INFO:tasks.rados.rados.0.smithi017.stdout:187: oid 3 updating version 0 to 57 2023-03-29T14:02:42.701 INFO:tasks.rados.rados.0.smithi017.stdout:187: oid 3 updating version 57 to 59 2023-03-29T14:02:42.701 INFO:tasks.rados.rados.0.smithi017.stdout:187: oid 3 version 59 is already newer than 59 2023-03-29T14:02:42.701 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 59 (ObjNum 119 snap 10 seq_num 119) dirty exists 2023-03-29T14:02:42.701 INFO:tasks.rados.rados.0.smithi017.stdout:187: left oid 3 (ObjNum 119 snap 10 seq_num 119) 2023-03-29T14:02:43.287 INFO:tasks.rados.rados.0.smithi017.stdout:184: done (4 left) 2023-03-29T14:02:43.287 INFO:tasks.rados.rados.0.smithi017.stdout:185: done (3 left) 2023-03-29T14:02:43.287 INFO:tasks.rados.rados.0.smithi017.stdout:187: done (2 left) 2023-03-29T14:02:43.287 INFO:tasks.rados.rados.0.smithi017.stdout:188: done (1 left) 2023-03-29T14:02:43.287 INFO:tasks.rados.rados.0.smithi017.stdout:189: done (0 left) 2023-03-29T14:02:43.287 INFO:tasks.rados.rados.0.smithi017.stdout:190: snap_create 2023-03-29T14:02:44.292 INFO:tasks.rados.rados.0.smithi017.stdout:190: done (0 left) 2023-03-29T14:02:44.292 INFO:tasks.rados.rados.0.smithi017.stdout:191: read oid 2 snap -1 2023-03-29T14:02:44.292 INFO:tasks.rados.rados.0.smithi017.stdout:191: expect (ObjNum 106 snap 9 seq_num 106) 2023-03-29T14:02:44.292 INFO:tasks.rados.rados.0.smithi017.stdout:192: snap_create 2023-03-29T14:02:44.411 INFO:tasks.rados.rados.0.smithi017.stdout:191: expect (ObjNum 106 snap 9 seq_num 106) 2023-03-29T14:02:44.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:44.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:44.822+0000 7fbe1e242700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:44.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:44.827+0000 7fbe1e242700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:45.291 INFO:tasks.rados.rados.0.smithi017.stdout:191: done (1 left) 2023-03-29T14:02:45.292 INFO:tasks.rados.rados.0.smithi017.stdout:192: done (0 left) 2023-03-29T14:02:45.292 INFO:tasks.rados.rados.0.smithi017.stdout:193: snap_create 2023-03-29T14:02:46.298 INFO:tasks.rados.rados.0.smithi017.stdout:193: done (0 left) 2023-03-29T14:02:46.299 INFO:tasks.rados.rados.0.smithi017.stdout:194: delete oid 46 current snap is 14 2023-03-29T14:02:46.403 INFO:tasks.rados.rados.0.smithi017.stdout:194: done (0 left) 2023-03-29T14:02:46.403 INFO:tasks.rados.rados.0.smithi017.stdout:195: snap_remove snap 13 2023-03-29T14:02:46.899 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:47.042+0000 7fe199e62700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:47.046 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:47.047+0000 7fe199e62700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:47.299 INFO:tasks.rados.rados.0.smithi017.stdout:195: done (0 left) 2023-03-29T14:02:47.299 INFO:tasks.rados.rados.0.smithi017.stdout:196: write (excl) oid 40 current snap is 14 2023-03-29T14:02:47.299 INFO:tasks.rados.rados.0.smithi017.stdout:196: seq_num 122 ranges {450454=642026,1505862=544592} 2023-03-29T14:02:47.305 INFO:tasks.rados.rados.0.smithi017.stdout:196: writing smithi017107884-40 from 450454 to 1092480 tid 1 2023-03-29T14:02:47.310 INFO:tasks.rados.rados.0.smithi017.stdout:196: writing smithi017107884-40 from 1505862 to 2050454 tid 2 2023-03-29T14:02:47.311 INFO:tasks.rados.rados.0.smithi017.stdout:197: read oid 8 snap 12 2023-03-29T14:02:47.311 INFO:tasks.rados.rados.0.smithi017.stdout:197: expect (ObjNum 7 snap 0 seq_num 7) 2023-03-29T14:02:47.311 INFO:tasks.rados.rados.0.smithi017.stdout:198: delete oid 17 current snap is 14 2023-03-29T14:02:47.347 INFO:tasks.rados.rados.0.smithi017.stdout:197: expect (ObjNum 7 snap 0 seq_num 7) 2023-03-29T14:02:47.434 INFO:tasks.rados.rados.0.smithi017.stdout:196: finishing write tid 1 to smithi017107884-40 2023-03-29T14:02:47.434 INFO:tasks.rados.rados.0.smithi017.stdout:197: done (2 left) 2023-03-29T14:02:47.434 INFO:tasks.rados.rados.0.smithi017.stdout:198: done (1 left) 2023-03-29T14:02:47.434 INFO:tasks.rados.rados.0.smithi017.stdout:199: snap_create 2023-03-29T14:02:47.441 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:47.442 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:196: finishing write tid 2 to smithi017107884-40 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:196: finishing write tid 3 to smithi017107884-40 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:196: finishing write tid 4 to smithi017107884-40 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:196: oid 40 updating version 0 to 32 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:196: oid 40 updating version 32 to 35 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:196: oid 40 version 35 is already newer than 35 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 35 (ObjNum 122 snap 14 seq_num 122) dirty exists 2023-03-29T14:02:47.482 INFO:tasks.rados.rados.0.smithi017.stdout:196: left oid 40 (ObjNum 122 snap 14 seq_num 122) 2023-03-29T14:02:47.576 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:47.571+0000 7fa5ed106700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:47.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:47.579+0000 7fa5ed106700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:48.306 INFO:tasks.rados.rados.0.smithi017.stdout:196: done (1 left) 2023-03-29T14:02:48.306 INFO:tasks.rados.rados.0.smithi017.stdout:199: done (0 left) 2023-03-29T14:02:48.306 INFO:tasks.rados.rados.0.smithi017.stdout:200: write (excl) oid 33 current snap is 15 2023-03-29T14:02:48.306 INFO:tasks.rados.rados.0.smithi017.stdout:200: seq_num 124 ranges {484068=792524,1906473=177595} 2023-03-29T14:02:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:02:48.313 INFO:tasks.rados.rados.0.smithi017.stdout:200: writing smithi017107884-33 from 484068 to 1276592 tid 1 2023-03-29T14:02:48.318 INFO:tasks.rados.rados.0.smithi017.stdout:200: writing smithi017107884-33 from 1906473 to 2084068 tid 2 2023-03-29T14:02:48.318 INFO:tasks.rados.rados.0.smithi017.stdout:201: read oid 46 snap 12 2023-03-29T14:02:48.318 INFO:tasks.rados.rados.0.smithi017.stdout:201: expect (ObjNum 62 snap 2 seq_num 62) 2023-03-29T14:02:48.318 INFO:tasks.rados.rados.0.smithi017.stdout:202: delete oid 19 current snap is 15 2023-03-29T14:02:48.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:02:48.429 INFO:tasks.rados.rados.0.smithi017.stdout:201: expect (ObjNum 62 snap 2 seq_num 62) 2023-03-29T14:02:48.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:48.502+0000 7f0df7004700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:48.511 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:48.512+0000 7f0df5da2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:48.604 INFO:tasks.rados.rados.0.smithi017.stdout:201: done (2 left) 2023-03-29T14:02:48.604 INFO:tasks.rados.rados.0.smithi017.stdout:202: done (1 left) 2023-03-29T14:02:48.604 INFO:tasks.rados.rados.0.smithi017.stdout:203: snap_remove snap 14 2023-03-29T14:02:49.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:49.249 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:49.248+0000 7fb0e6fa7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:49.253+0000 7fb0e6fa7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:49.309 INFO:tasks.rados.rados.0.smithi017.stdout:203: done (1 left) 2023-03-29T14:02:49.309 INFO:tasks.rados.rados.0.smithi017.stdout:204: read oid 28 snap -1 2023-03-29T14:02:49.309 INFO:tasks.rados.rados.0.smithi017.stdout:204: expect (ObjNum 78 snap 6 seq_num 78) 2023-03-29T14:02:49.309 INFO:tasks.rados.rados.0.smithi017.stdout:205: write (excl) oid 17 current snap is 15 2023-03-29T14:02:49.309 INFO:tasks.rados.rados.0.smithi017.stdout:200: finishing write tid 1 to smithi017107884-33 2023-03-29T14:02:49.309 INFO:tasks.rados.rados.0.smithi017.stdout:200: finishing write tid 2 to smithi017107884-33 2023-03-29T14:02:49.309 INFO:tasks.rados.rados.0.smithi017.stdout:205: seq_num 126 ranges {517682=543022,1623437=494245} 2023-03-29T14:02:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:02:49.314 INFO:tasks.rados.rados.0.smithi017.stdout:205: writing smithi017107884-17 from 517682 to 1060704 tid 1 2023-03-29T14:02:49.321 INFO:tasks.rados.rados.0.smithi017.stdout:205: writing smithi017107884-17 from 1623437 to 2117682 tid 2 2023-03-29T14:02:49.321 INFO:tasks.rados.rados.0.smithi017.stdout:206: read oid 48 snap -1 2023-03-29T14:02:49.321 INFO:tasks.rados.rados.0.smithi017.stdout:206: expect deleted 2023-03-29T14:02:49.321 INFO:tasks.rados.rados.0.smithi017.stdout:207: snap_remove snap 12 2023-03-29T14:02:49.325 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:02:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:49.498+0000 7fd5e6398700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:49.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:49.507+0000 7fd5e6398700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:207: done (4 left) 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:208: read oid 18 snap -1 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:208: expect (ObjNum 17 snap 0 seq_num 17) 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:209: write (excl) oid 36 current snap is 15 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:200: finishing write tid 3 to smithi017107884-33 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:200: finishing write tid 4 to smithi017107884-33 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:200: oid 33 updating version 0 to 26 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:200: oid 33 version 26 is already newer than 23 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:200: oid 33 version 26 is already newer than 26 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 26 (ObjNum 124 snap 15 seq_num 124) dirty exists 2023-03-29T14:02:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:200: left oid 33 (ObjNum 124 snap 15 seq_num 124) 2023-03-29T14:02:50.316 INFO:tasks.rados.rados.0.smithi017.stdout:209: seq_num 127 ranges {534489=618271,1565566=568923} 2023-03-29T14:02:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:02:50.321 INFO:tasks.rados.rados.0.smithi017.stdout:209: writing smithi017107884-36 from 534489 to 1152760 tid 1 2023-03-29T14:02:50.327 INFO:tasks.rados.rados.0.smithi017.stdout:209: writing smithi017107884-36 from 1565566 to 2134489 tid 2 2023-03-29T14:02:50.327 INFO:tasks.rados.rados.0.smithi017.stdout:200: done (5 left) 2023-03-29T14:02:50.327 INFO:tasks.rados.rados.0.smithi017.stdout:210: read oid 21 snap -1 2023-03-29T14:02:50.327 INFO:tasks.rados.rados.0.smithi017.stdout:210: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:02:50.327 INFO:tasks.rados.rados.0.smithi017.stdout:211: delete oid 49 current snap is 15 2023-03-29T14:02:50.327 INFO:tasks.rados.rados.0.smithi017.stdout:204: expect (ObjNum 78 snap 6 seq_num 78) 2023-03-29T14:02:50.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:02:50.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:50.470+0000 7f60d955d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:50.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:50.478+0000 7f60d955d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:50.549 INFO:tasks.rados.rados.0.smithi017.stdout:205: finishing write tid 1 to smithi017107884-17 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:205: finishing write tid 2 to smithi017107884-17 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:205: finishing write tid 3 to smithi017107884-17 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:204: done (6 left) 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:211: done (5 left) 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:212: read oid 29 snap -1 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:212: expect (ObjNum 90 snap 7 seq_num 90) 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:213: write oid 37 current snap is 15 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:205: finishing write tid 4 to smithi017107884-17 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:205: oid 17 updating version 0 to 43 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:205: oid 17 updating version 43 to 45 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:205: oid 17 version 45 is already newer than 45 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 45 (ObjNum 126 snap 15 seq_num 126) dirty exists 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:205: left oid 17 (ObjNum 126 snap 15 seq_num 126) 2023-03-29T14:02:50.550 INFO:tasks.rados.rados.0.smithi017.stdout:213: seq_num 129 ranges {568103=768769,2082530=85573} 2023-03-29T14:02:50.559 INFO:tasks.rados.rados.0.smithi017.stdout:213: writing smithi017107884-37 from 568103 to 1336872 tid 1 2023-03-29T14:02:50.564 INFO:tasks.rados.rados.0.smithi017.stdout:213: writing smithi017107884-37 from 2082530 to 2168103 tid 2 2023-03-29T14:02:50.564 INFO:tasks.rados.rados.0.smithi017.stdout:205: done (6 left) 2023-03-29T14:02:50.564 INFO:tasks.rados.rados.0.smithi017.stdout:206: done (5 left) 2023-03-29T14:02:50.564 INFO:tasks.rados.rados.0.smithi017.stdout:214: write oid 23 current snap is 15 2023-03-29T14:02:50.564 INFO:tasks.rados.rados.0.smithi017.stdout:214: seq_num 130 ranges {584910=560371,1857365=327545} 2023-03-29T14:02:50.570 INFO:tasks.rados.rados.0.smithi017.stdout:214: writing smithi017107884-23 from 584910 to 1145281 tid 1 2023-03-29T14:02:50.576 INFO:tasks.rados.rados.0.smithi017.stdout:214: writing smithi017107884-23 from 1857365 to 2184910 tid 2 2023-03-29T14:02:50.577 INFO:tasks.rados.rados.0.smithi017.stdout:215: snap_remove snap 10 2023-03-29T14:02:51.316 INFO:tasks.rados.rados.0.smithi017.stdout:215: done (6 left) 2023-03-29T14:02:51.316 INFO:tasks.rados.rados.0.smithi017.stdout:216: delete oid 42 current snap is 15 2023-03-29T14:02:51.316 INFO:tasks.rados.rados.0.smithi017.stdout:208: expect (ObjNum 17 snap 0 seq_num 17) 2023-03-29T14:02:51.319 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:02:51.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:51.332 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:02:51.333 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:51.407 INFO:tasks.rados.rados.0.smithi017.stdout:210: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:02:51.464 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:51.463+0000 7f91a68fd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:51.467 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:51.468+0000 7f91a68fd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:51.501 INFO:tasks.rados.rados.0.smithi017.stdout:209: finishing write tid 1 to smithi017107884-36 2023-03-29T14:02:51.501 INFO:tasks.rados.rados.0.smithi017.stdout:209: finishing write tid 2 to smithi017107884-36 2023-03-29T14:02:51.501 INFO:tasks.rados.rados.0.smithi017.stdout:209: finishing write tid 3 to smithi017107884-36 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:209: finishing write tid 4 to smithi017107884-36 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:209: oid 36 updating version 0 to 32 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:209: oid 36 version 32 is already newer than 30 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:209: oid 36 version 32 is already newer than 32 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 32 (ObjNum 127 snap 15 seq_num 127) dirty exists 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:209: left oid 36 (ObjNum 127 snap 15 seq_num 127) 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:208: done (6 left) 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:209: done (5 left) 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:210: done (4 left) 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:216: done (3 left) 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:217: write (excl) oid 38 current snap is 15 2023-03-29T14:02:51.502 INFO:tasks.rados.rados.0.smithi017.stdout:212: expect (ObjNum 90 snap 7 seq_num 90) 2023-03-29T14:02:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:51.520+0000 7fcd9173a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:51.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:51.526+0000 7fcd9173a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:51.686 INFO:tasks.rados.rados.0.smithi017.stdout:213: finishing write tid 1 to smithi017107884-37 2023-03-29T14:02:51.686 INFO:tasks.rados.rados.0.smithi017.stdout:213: finishing write tid 2 to smithi017107884-37 2023-03-29T14:02:51.686 INFO:tasks.rados.rados.0.smithi017.stdout:217: seq_num 132 ranges {618524=710869,1857976=360548} 2023-03-29T14:02:51.698 INFO:tasks.rados.rados.0.smithi017.stdout:217: writing smithi017107884-38 from 618524 to 1329393 tid 1 2023-03-29T14:02:51.705 INFO:tasks.rados.rados.0.smithi017.stdout:217: writing smithi017107884-38 from 1857976 to 2218524 tid 2 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:212: done (3 left) 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:218: delete oid 49 current snap is 15 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:213: finishing write tid 3 to smithi017107884-37 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:213: finishing write tid 4 to smithi017107884-37 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:213: oid 37 updating version 0 to 13 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:213: oid 37 version 13 is already newer than 12 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:213: oid 37 version 13 is already newer than 13 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 13 (ObjNum 129 snap 15 seq_num 129) dirty exists 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:213: left oid 37 (ObjNum 129 snap 15 seq_num 129) 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:214: finishing write tid 1 to smithi017107884-23 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:214: finishing write tid 2 to smithi017107884-23 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:214: finishing write tid 3 to smithi017107884-23 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:214: finishing write tid 4 to smithi017107884-23 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:214: oid 23 updating version 0 to 63 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:214: oid 23 version 63 is already newer than 63 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:214: oid 23 version 63 is already newer than 63 2023-03-29T14:02:51.706 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 63 (ObjNum 130 snap 15 seq_num 130) dirty exists 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:214: left oid 23 (ObjNum 130 snap 15 seq_num 130) 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:213: done (3 left) 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:214: done (2 left) 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:218: done (1 left) 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:219: read oid 16 snap -1 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:219: expect (ObjNum 15 snap 0 seq_num 15) 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:220: write (excl) oid 15 current snap is 15 2023-03-29T14:02:51.707 INFO:tasks.rados.rados.0.smithi017.stdout:220: seq_num 134 ranges {652138=461367,1574940=677198} 2023-03-29T14:02:51.717 INFO:tasks.rados.rados.0.smithi017.stdout:220: writing smithi017107884-15 from 652138 to 1113505 tid 1 2023-03-29T14:02:51.727 INFO:tasks.rados.rados.0.smithi017.stdout:220: writing smithi017107884-15 from 1574940 to 2252138 tid 2 2023-03-29T14:02:51.727 INFO:tasks.rados.rados.0.smithi017.stdout:221: snap_create 2023-03-29T14:02:51.761 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:51.766 INFO:tasks.rados.rados.0.smithi017.stdout:219: expect (ObjNum 15 snap 0 seq_num 15) 2023-03-29T14:02:51.783 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-03-29T14:02:51.313279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":246930446,"num_objects":148,"num_object_clones":54,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":768,"num_read_kb":286010,"num_write":1206,"num_write_kb":272844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":510,"ondisk_log_size":510,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":317396,"kb_used_data":292020,"kb_used_omap":0,"kb_used_meta":25344,"kb_avail":377169964,"statfs":{"total":386547056640,"available":386222043136,"internally_reserved":0,"allocated":299028480,"data_stored":315493516,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":25952256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4907380,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":32440,"num_write":40,"num_write_kb":10298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015211"},"pg_stats":[{"pgid":"3.f","version":"21'14","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-29T14:02:45.342595+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:45.342595+0000","last_peered":"2023-03-29T14:02:45.342595+0000","last_clean":"2023-03-29T14:02:45.342595+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:45.342595+0000","last_undegraded":"2023-03-29T14:02:45.342595+0000","last_fullsized":"2023-03-29T14:02:45.342595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10890804,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":24329,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"17'3","reported_seq":37,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.492477+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:49.492477+0000","last_peered":"2023-03-29T14:02:49.492477+0000","last_clean":"2023-03-29T14:02:49.492477+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:49.492477+0000","last_undegraded":"2023-03-29T14:02:49.492477+0000","last_fullsized":"2023-03-29T14:02:49.492477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243718,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"44'45","reported_seq":127,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.451304+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:49.451304+0000","last_peered":"2023-03-29T14:02:49.451304+0000","last_clean":"2023-03-29T14:02:49.451304+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:49.451304+0000","last_undegraded":"2023-03-29T14:02:49.451304+0000","last_fullsized":"2023-03-29T14:02:49.451304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832128,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":7053,"num_write":98,"num_write_kb":28044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"42'35","reported_seq":113,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.350276+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:49.350276+0000","last_peered":"2023-03-29T14:02:49.350276+0000","last_clean":"2023-03-29T14:02:49.350276+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:49.350276+0000","last_undegraded":"2023-03-29T14:02:49.350276+0000","last_fullsized":"2023-03-29T14:02:49.350276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143008,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":15162,"num_write":80,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"41'36","reported_seq":121,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.455905+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:49.455905+0000","last_peered":"2023-03-29T14:02:49.455905+0000","last_clean":"2023-03-29T14:02:49.455905+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:49.455905+0000","last_undegraded":"2023-03-29T14:02:49.455905+0000","last_fullsized":"2023-03-29T14:02:49.455905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22890936,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":63,"num_read_kb":34881,"num_write":80,"num_write_kb":23830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.345678+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:49.345678+0000","last_peered":"2023-03-29T14:02:49.345678+0000","last_clean":"2023-03-29T14:02:49.345678+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:49.345678+0000","last_undegraded":"2023-03-29T14:02:49.345678+0000","last_fullsized":"2023-03-29T14:02:49.345678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"31'18","reported_seq":76,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.368216+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:50.368216+0000","last_peered":"2023-03-29T14:02:50.368216+0000","last_clean":"2023-03-29T14:02:50.368216+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:50.368216+0000","last_undegraded":"2023-03-29T14:02:50.368216+0000","last_fullsized":"2023-03-29T14:02:50.368216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16672346,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":10583,"num_write":42,"num_write_kb":11466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.372577+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:49.372577+0000","last_peered":"2023-03-29T14:02:49.372577+0000","last_clean":"2023-03-29T14:02:49.372577+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:49.372577+0000","last_undegraded":"2023-03-29T14:02:49.372577+0000","last_fullsized":"2023-03-29T14:02:49.372577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":20934,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.458746+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:49.458746+0000","last_peered":"2023-03-29T14:02:49.458746+0000","last_clean":"2023-03-29T14:02:49.458746+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:49.458746+0000","last_undegraded":"2023-03-29T14:02:49.458746+0000","last_fullsized":"2023-03-29T14:02:49.458746+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"37'9","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-29T14:02:45.349734+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:45.349734+0000","last_peered":"2023-03-29T14:02:45.349734+0000","last_clean":"2023-03-29T14:02:45.349734+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:45.349734+0000","last_undegraded":"2023-03-29T14:02:45.349734+0000","last_fullsized":"2023-03-29T14:02:45.349734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210104,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13755,"num_write":16,"num_write_kb":4726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.343264+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:49.343264+0000","last_peered":"2023-03-29T14:02:49.343264+0000","last_clean":"2023-03-29T14:02:49.343264+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:49.343264+0000","last_undegraded":"2023-03-29T14:02:49.343264+0000","last_fullsized":"2023-03-29T14:02:49.343264+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.563776+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:50.563776+0000","last_peered":"2023-03-29T14:02:50.563776+0000","last_clean":"2023-03-29T14:02:50.563776+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:50.563776+0000","last_undegraded":"2023-03-29T14:02:50.563776+0000","last_fullsized":"2023-03-29T14:02:50.563776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":14439,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-29T14:02:45.335753+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:45.335753+0000","last_peered":"2023-03-29T14:02:45.335753+0000","last_clean":"2023-03-29T14:02:45.335753+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:45.335753+0000","last_undegraded":"2023-03-29T14:02:45.335753+0000","last_fullsized":"2023-03-29T14:02:45.335753+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"45'32","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.440333+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:50.440333+0000","last_peered":"2023-03-29T14:02:50.440333+0000","last_clean":"2023-03-29T14:02:50.440333+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:50.440333+0000","last_undegraded":"2023-03-29T14:02:50.440333+0000","last_fullsized":"2023-03-29T14:02:50.440333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17848902,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":23602,"num_write":72,"num_write_kb":22142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.343841+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:49.343841+0000","last_peered":"2023-03-29T14:02:49.343841+0000","last_clean":"2023-03-29T14:02:49.343841+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:49.343841+0000","last_undegraded":"2023-03-29T14:02:49.343841+0000","last_fullsized":"2023-03-29T14:02:49.343841+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":258,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.484234+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:49.484234+0000","last_peered":"2023-03-29T14:02:49.484234+0000","last_clean":"2023-03-29T14:02:49.484234+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:49.484234+0000","last_undegraded":"2023-03-29T14:02:49.484234+0000","last_fullsized":"2023-03-29T14:02:49.484234+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"37'40","reported_seq":133,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.378108+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:50.378108+0000","last_peered":"2023-03-29T14:02:50.378108+0000","last_clean":"2023-03-29T14:02:50.378108+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:50.378108+0000","last_undegraded":"2023-03-29T14:02:50.378108+0000","last_fullsized":"2023-03-29T14:02:50.378108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21411920,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":32183,"num_write":90,"num_write_kb":26440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.334493+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:49.334493+0000","last_peered":"2023-03-29T14:02:49.334493+0000","last_clean":"2023-03-29T14:02:49.334493+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:49.334493+0000","last_undegraded":"2023-03-29T14:02:49.334493+0000","last_fullsized":"2023-03-29T14:02:49.334493+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"45'63","reported_seq":183,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.600655+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:50.600655+0000","last_peered":"2023-03-29T14:02:50.600655+0000","last_clean":"2023-03-29T14:02:50.600655+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:50.600655+0000","last_undegraded":"2023-03-29T14:02:50.600655+0000","last_fullsized":"2023-03-29T14:02:50.600655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28605316,"num_objects":16,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":39000,"num_write":140,"num_write_kb":40352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.463476+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:49.463476+0000","last_peered":"2023-03-29T14:02:49.463476+0000","last_clean":"2023-03-29T14:02:49.463476+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:49.463476+0000","last_undegraded":"2023-03-29T14:02:49.463476+0000","last_fullsized":"2023-03-29T14:02:49.463476+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.475188+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:49.475188+0000","last_peered":"2023-03-29T14:02:49.475188+0000","last_clean":"2023-03-29T14:02:49.475188+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:49.475188+0000","last_undegraded":"2023-03-29T14:02:49.475188+0000","last_fullsized":"2023-03-29T14:02:49.475188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":15525,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.459436+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:49.459436+0000","last_peered":"2023-03-29T14:02:49.459436+0000","last_clean":"2023-03-29T14:02:49.459436+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:49.459436+0000","last_undegraded":"2023-03-29T14:02:49.459436+0000","last_fullsized":"2023-03-29T14:02:49.459436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":11149,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.479194+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:49.479194+0000","last_peered":"2023-03-29T14:02:49.479194+0000","last_clean":"2023-03-29T14:02:49.479194+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:49.479194+0000","last_undegraded":"2023-03-29T14:02:49.479194+0000","last_fullsized":"2023-03-29T14:02:49.479194+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.340184+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:49.340184+0000","last_peered":"2023-03-29T14:02:49.340184+0000","last_clean":"2023-03-29T14:02:49.340184+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:49.340184+0000","last_undegraded":"2023-03-29T14:02:49.340184+0000","last_fullsized":"2023-03-29T14:02:49.340184+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"43'27","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.450371+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:49.450371+0000","last_peered":"2023-03-29T14:02:49.450371+0000","last_clean":"2023-03-29T14:02:49.450371+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:49.450371+0000","last_undegraded":"2023-03-29T14:02:49.450371+0000","last_fullsized":"2023-03-29T14:02:49.450371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630378,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":23294,"num_write":60,"num_write_kb":16304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":245749710,"num_objects":144,"num_object_clones":54,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":626,"num_read_kb":285890,"num_write":932,"num_write_kb":269056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":413,"ondisk_log_size":413,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86616,"kb_used_data":80464,"kb_used_omap":0,"kb_used_meta":6144,"kb_avail":94285224,"statfs":{"total":96636764160,"available":96548069376,"internally_reserved":0,"allocated":82395136,"data_stored":87300690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6291456},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50820,"kb_used_data":44732,"kb_used_omap":0,"kb_used_meta":6080,"kb_avail":94321020,"statfs":{"total":96636764160,"available":96584724480,"internally_reserved":0,"allocated":45805568,"data_stored":47053486,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6225920},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91504,"kb_used_data":84840,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94280336,"statfs":{"total":96636764160,"available":96543064064,"internally_reserved":0,"allocated":86876160,"data_stored":92333897,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88456,"kb_used_data":81984,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94283384,"statfs":{"total":96636764160,"available":96546185216,"internally_reserved":0,"allocated":83951616,"data_stored":88805443,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:51.786 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: finishing write tid 1 to smithi017107884-15 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:217: finishing write tid 1 to smithi017107884-38 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: finishing write tid 2 to smithi017107884-15 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: finishing write tid 3 to smithi017107884-15 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: finishing write tid 4 to smithi017107884-15 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: oid 15 updating version 0 to 68 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: oid 15 version 68 is already newer than 65 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: oid 15 version 68 is already newer than 68 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 68 (ObjNum 134 snap 15 seq_num 134) dirty exists 2023-03-29T14:02:51.857 INFO:tasks.rados.rados.0.smithi017.stdout:220: left oid 15 (ObjNum 134 snap 15 seq_num 134) 2023-03-29T14:02:51.867 INFO:tasks.rados.rados.0.smithi017.stdout:217: finishing write tid 2 to smithi017107884-38 2023-03-29T14:02:51.868 INFO:tasks.rados.rados.0.smithi017.stdout:217: finishing write tid 3 to smithi017107884-38 2023-03-29T14:02:51.869 INFO:tasks.rados.rados.0.smithi017.stdout:217: finishing write tid 4 to smithi017107884-38 2023-03-29T14:02:51.869 INFO:tasks.rados.rados.0.smithi017.stdout:217: oid 38 updating version 0 to 7 2023-03-29T14:02:51.869 INFO:tasks.rados.rados.0.smithi017.stdout:217: oid 38 version 7 is already newer than 7 2023-03-29T14:02:51.869 INFO:tasks.rados.rados.0.smithi017.stdout:217: oid 38 version 7 is already newer than 4 2023-03-29T14:02:51.869 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 7 (ObjNum 132 snap 15 seq_num 132) dirty exists 2023-03-29T14:02:51.869 INFO:tasks.rados.rados.0.smithi017.stdout:217: left oid 38 (ObjNum 132 snap 15 seq_num 132) 2023-03-29T14:02:51.920 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:51.917+0000 7fd758fb4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:51.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:51.923+0000 7fd758fb4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:52.130 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:52.142 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-03-29T14:02:52.103201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":246930446,"num_objects":148,"num_object_clones":54,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":768,"num_read_kb":286010,"num_write":1206,"num_write_kb":272844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":510,"ondisk_log_size":510,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":317396,"kb_used_data":292020,"kb_used_omap":0,"kb_used_meta":25344,"kb_avail":377169964,"statfs":{"total":386547056640,"available":386222043136,"internally_reserved":0,"allocated":299028480,"data_stored":315493516,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":25952256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4907380,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":32440,"num_write":40,"num_write_kb":10298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000816"},"pg_stats":[{"pgid":"3.f","version":"21'14","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-29T14:02:45.342595+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:45.342595+0000","last_peered":"2023-03-29T14:02:45.342595+0000","last_clean":"2023-03-29T14:02:45.342595+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:45.342595+0000","last_undegraded":"2023-03-29T14:02:45.342595+0000","last_fullsized":"2023-03-29T14:02:45.342595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10890804,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":24329,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"17'3","reported_seq":37,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.492477+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:49.492477+0000","last_peered":"2023-03-29T14:02:49.492477+0000","last_clean":"2023-03-29T14:02:49.492477+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:49.492477+0000","last_undegraded":"2023-03-29T14:02:49.492477+0000","last_fullsized":"2023-03-29T14:02:49.492477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243718,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"44'45","reported_seq":127,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.451304+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:49.451304+0000","last_peered":"2023-03-29T14:02:49.451304+0000","last_clean":"2023-03-29T14:02:49.451304+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:49.451304+0000","last_undegraded":"2023-03-29T14:02:49.451304+0000","last_fullsized":"2023-03-29T14:02:49.451304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832128,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":7053,"num_write":98,"num_write_kb":28044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"42'35","reported_seq":113,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.350276+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:49.350276+0000","last_peered":"2023-03-29T14:02:49.350276+0000","last_clean":"2023-03-29T14:02:49.350276+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:49.350276+0000","last_undegraded":"2023-03-29T14:02:49.350276+0000","last_fullsized":"2023-03-29T14:02:49.350276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143008,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":15162,"num_write":80,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"41'36","reported_seq":121,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.455905+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:49.455905+0000","last_peered":"2023-03-29T14:02:49.455905+0000","last_clean":"2023-03-29T14:02:49.455905+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:49.455905+0000","last_undegraded":"2023-03-29T14:02:49.455905+0000","last_fullsized":"2023-03-29T14:02:49.455905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22890936,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":63,"num_read_kb":34881,"num_write":80,"num_write_kb":23830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.345678+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:49.345678+0000","last_peered":"2023-03-29T14:02:49.345678+0000","last_clean":"2023-03-29T14:02:49.345678+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:49.345678+0000","last_undegraded":"2023-03-29T14:02:49.345678+0000","last_fullsized":"2023-03-29T14:02:49.345678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"31'18","reported_seq":76,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.368216+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:50.368216+0000","last_peered":"2023-03-29T14:02:50.368216+0000","last_clean":"2023-03-29T14:02:50.368216+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:50.368216+0000","last_undegraded":"2023-03-29T14:02:50.368216+0000","last_fullsized":"2023-03-29T14:02:50.368216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16672346,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":10583,"num_write":42,"num_write_kb":11466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.372577+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:49.372577+0000","last_peered":"2023-03-29T14:02:49.372577+0000","last_clean":"2023-03-29T14:02:49.372577+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:49.372577+0000","last_undegraded":"2023-03-29T14:02:49.372577+0000","last_fullsized":"2023-03-29T14:02:49.372577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":20934,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.458746+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:49.458746+0000","last_peered":"2023-03-29T14:02:49.458746+0000","last_clean":"2023-03-29T14:02:49.458746+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:49.458746+0000","last_undegraded":"2023-03-29T14:02:49.458746+0000","last_fullsized":"2023-03-29T14:02:49.458746+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"37'9","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-29T14:02:45.349734+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:45.349734+0000","last_peered":"2023-03-29T14:02:45.349734+0000","last_clean":"2023-03-29T14:02:45.349734+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:45.349734+0000","last_undegraded":"2023-03-29T14:02:45.349734+0000","last_fullsized":"2023-03-29T14:02:45.349734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210104,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":13755,"num_write":16,"num_write_kb":4726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.343264+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:49.343264+0000","last_peered":"2023-03-29T14:02:49.343264+0000","last_clean":"2023-03-29T14:02:49.343264+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:49.343264+0000","last_undegraded":"2023-03-29T14:02:49.343264+0000","last_fullsized":"2023-03-29T14:02:49.343264+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.563776+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:50.563776+0000","last_peered":"2023-03-29T14:02:50.563776+0000","last_clean":"2023-03-29T14:02:50.563776+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:50.563776+0000","last_undegraded":"2023-03-29T14:02:50.563776+0000","last_fullsized":"2023-03-29T14:02:50.563776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":14439,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-29T14:02:45.335753+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:45.335753+0000","last_peered":"2023-03-29T14:02:45.335753+0000","last_clean":"2023-03-29T14:02:45.335753+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:45.335753+0000","last_undegraded":"2023-03-29T14:02:45.335753+0000","last_fullsized":"2023-03-29T14:02:45.335753+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"45'32","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.440333+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:50.440333+0000","last_peered":"2023-03-29T14:02:50.440333+0000","last_clean":"2023-03-29T14:02:50.440333+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:50.440333+0000","last_undegraded":"2023-03-29T14:02:50.440333+0000","last_fullsized":"2023-03-29T14:02:50.440333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17848902,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":23602,"num_write":72,"num_write_kb":22142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.343841+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:49.343841+0000","last_peered":"2023-03-29T14:02:49.343841+0000","last_clean":"2023-03-29T14:02:49.343841+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:49.343841+0000","last_undegraded":"2023-03-29T14:02:49.343841+0000","last_fullsized":"2023-03-29T14:02:49.343841+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":258,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.484234+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:49.484234+0000","last_peered":"2023-03-29T14:02:49.484234+0000","last_clean":"2023-03-29T14:02:49.484234+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:49.484234+0000","last_undegraded":"2023-03-29T14:02:49.484234+0000","last_fullsized":"2023-03-29T14:02:49.484234+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"37'40","reported_seq":133,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.378108+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:50.378108+0000","last_peered":"2023-03-29T14:02:50.378108+0000","last_clean":"2023-03-29T14:02:50.378108+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:50.378108+0000","last_undegraded":"2023-03-29T14:02:50.378108+0000","last_fullsized":"2023-03-29T14:02:50.378108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21411920,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":32183,"num_write":90,"num_write_kb":26440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.334493+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:49.334493+0000","last_peered":"2023-03-29T14:02:49.334493+0000","last_clean":"2023-03-29T14:02:49.334493+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:49.334493+0000","last_undegraded":"2023-03-29T14:02:49.334493+0000","last_fullsized":"2023-03-29T14:02:49.334493+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"45'63","reported_seq":183,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.600655+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:50.600655+0000","last_peered":"2023-03-29T14:02:50.600655+0000","last_clean":"2023-03-29T14:02:50.600655+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:50.600655+0000","last_undegraded":"2023-03-29T14:02:50.600655+0000","last_fullsized":"2023-03-29T14:02:50.600655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28605316,"num_objects":16,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":39000,"num_write":140,"num_write_kb":40352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.463476+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:49.463476+0000","last_peered":"2023-03-29T14:02:49.463476+0000","last_clean":"2023-03-29T14:02:49.463476+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:49.463476+0000","last_undegraded":"2023-03-29T14:02:49.463476+0000","last_fullsized":"2023-03-29T14:02:49.463476+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.475188+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:49.475188+0000","last_peered":"2023-03-29T14:02:49.475188+0000","last_clean":"2023-03-29T14:02:49.475188+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:49.475188+0000","last_undegraded":"2023-03-29T14:02:49.475188+0000","last_fullsized":"2023-03-29T14:02:49.475188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":15525,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.459436+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:49.459436+0000","last_peered":"2023-03-29T14:02:49.459436+0000","last_clean":"2023-03-29T14:02:49.459436+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:49.459436+0000","last_undegraded":"2023-03-29T14:02:49.459436+0000","last_fullsized":"2023-03-29T14:02:49.459436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":11149,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.479194+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:49.479194+0000","last_peered":"2023-03-29T14:02:49.479194+0000","last_clean":"2023-03-29T14:02:49.479194+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:49.479194+0000","last_undegraded":"2023-03-29T14:02:49.479194+0000","last_fullsized":"2023-03-29T14:02:49.479194+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.340184+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:49.340184+0000","last_peered":"2023-03-29T14:02:49.340184+0000","last_clean":"2023-03-29T14:02:49.340184+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:49.340184+0000","last_undegraded":"2023-03-29T14:02:49.340184+0000","last_fullsized":"2023-03-29T14:02:49.340184+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"43'27","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.450371+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:49.450371+0000","last_peered":"2023-03-29T14:02:49.450371+0000","last_clean":"2023-03-29T14:02:49.450371+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:49.450371+0000","last_undegraded":"2023-03-29T14:02:49.450371+0000","last_fullsized":"2023-03-29T14:02:49.450371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630378,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":23294,"num_write":60,"num_write_kb":16304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":245749710,"num_objects":144,"num_object_clones":54,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":626,"num_read_kb":285890,"num_write":932,"num_write_kb":269056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":413,"ondisk_log_size":413,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86616,"kb_used_data":80464,"kb_used_omap":0,"kb_used_meta":6144,"kb_avail":94285224,"statfs":{"total":96636764160,"available":96548069376,"internally_reserved":0,"allocated":82395136,"data_stored":87300690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6291456},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50820,"kb_used_data":44732,"kb_used_omap":0,"kb_used_meta":6080,"kb_avail":94321020,"statfs":{"total":96636764160,"available":96584724480,"internally_reserved":0,"allocated":45805568,"data_stored":47053486,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6225920},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91504,"kb_used_data":84840,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94280336,"statfs":{"total":96636764160,"available":96543064064,"internally_reserved":0,"allocated":86876160,"data_stored":92333897,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88456,"kb_used_data":81984,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94283384,"statfs":{"total":96636764160,"available":96546185216,"internally_reserved":0,"allocated":83951616,"data_stored":88805443,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:52.146 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:02:52.146 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:52.146 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:52.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:52.275+0000 7fb7fca5d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:52.282+0000 7fb7fca5d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:52.321 INFO:tasks.rados.rados.0.smithi017.stdout:217: done (3 left) 2023-03-29T14:02:52.322 INFO:tasks.rados.rados.0.smithi017.stdout:219: done (2 left) 2023-03-29T14:02:52.322 INFO:tasks.rados.rados.0.smithi017.stdout:220: done (1 left) 2023-03-29T14:02:52.322 INFO:tasks.rados.rados.0.smithi017.stdout:221: done (0 left) 2023-03-29T14:02:52.322 INFO:tasks.rados.rados.0.smithi017.stdout:222: snap_create 2023-03-29T14:02:52.485 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:52.487 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:52.500 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-03-29T14:02:52.320656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":246056548,"num_objects":150,"num_object_clones":56,"num_object_copies":450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":776,"num_read_kb":292611,"num_write":1214,"num_write_kb":274514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":514,"ondisk_log_size":514,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":323488,"kb_used_data":298048,"kb_used_omap":0,"kb_used_meta":25408,"kb_avail":377163872,"statfs":{"total":386547056640,"available":386215804928,"internally_reserved":0,"allocated":305201152,"data_stored":322015257,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":26017792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5781278,"num_objects":12,"num_object_clones":10,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":39041,"num_write":48,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016832"},"pg_stats":[{"pgid":"3.f","version":"21'14","reported_seq":76,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-03-29T14:02:51.761840+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:51.761840+0000","last_peered":"2023-03-29T14:02:51.761840+0000","last_clean":"2023-03-29T14:02:51.761840+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:51.761840+0000","last_undegraded":"2023-03-29T14:02:51.761840+0000","last_fullsized":"2023-03-29T14:02:51.761840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10890804,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":30929,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"17'3","reported_seq":37,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.492477+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:49.492477+0000","last_peered":"2023-03-29T14:02:49.492477+0000","last_clean":"2023-03-29T14:02:49.492477+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:49.492477+0000","last_undegraded":"2023-03-29T14:02:49.492477+0000","last_fullsized":"2023-03-29T14:02:49.492477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243718,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"44'45","reported_seq":127,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.451304+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:49.451304+0000","last_peered":"2023-03-29T14:02:49.451304+0000","last_clean":"2023-03-29T14:02:49.451304+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:49.451304+0000","last_undegraded":"2023-03-29T14:02:49.451304+0000","last_fullsized":"2023-03-29T14:02:49.451304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832128,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":7053,"num_write":98,"num_write_kb":28044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"42'35","reported_seq":113,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.350276+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:49.350276+0000","last_peered":"2023-03-29T14:02:49.350276+0000","last_clean":"2023-03-29T14:02:49.350276+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:49.350276+0000","last_undegraded":"2023-03-29T14:02:49.350276+0000","last_fullsized":"2023-03-29T14:02:49.350276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143008,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":15162,"num_write":80,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"41'36","reported_seq":121,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.455905+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:49.455905+0000","last_peered":"2023-03-29T14:02:49.455905+0000","last_clean":"2023-03-29T14:02:49.455905+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:49.455905+0000","last_undegraded":"2023-03-29T14:02:49.455905+0000","last_fullsized":"2023-03-29T14:02:49.455905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22890936,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":63,"num_read_kb":34881,"num_write":80,"num_write_kb":23830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.345678+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:49.345678+0000","last_peered":"2023-03-29T14:02:49.345678+0000","last_clean":"2023-03-29T14:02:49.345678+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:49.345678+0000","last_undegraded":"2023-03-29T14:02:49.345678+0000","last_fullsized":"2023-03-29T14:02:49.345678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"31'18","reported_seq":76,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.368216+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:50.368216+0000","last_peered":"2023-03-29T14:02:50.368216+0000","last_clean":"2023-03-29T14:02:50.368216+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:50.368216+0000","last_undegraded":"2023-03-29T14:02:50.368216+0000","last_fullsized":"2023-03-29T14:02:50.368216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16672346,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":10583,"num_write":42,"num_write_kb":11466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.372577+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:49.372577+0000","last_peered":"2023-03-29T14:02:49.372577+0000","last_clean":"2023-03-29T14:02:49.372577+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:49.372577+0000","last_undegraded":"2023-03-29T14:02:49.372577+0000","last_fullsized":"2023-03-29T14:02:49.372577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":20934,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.458746+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:49.458746+0000","last_peered":"2023-03-29T14:02:49.458746+0000","last_clean":"2023-03-29T14:02:49.458746+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:49.458746+0000","last_undegraded":"2023-03-29T14:02:49.458746+0000","last_fullsized":"2023-03-29T14:02:49.458746+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"45'13","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.652875+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:50.652875+0000","last_peered":"2023-03-29T14:02:50.652875+0000","last_clean":"2023-03-29T14:02:50.652875+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:50.652875+0000","last_undegraded":"2023-03-29T14:02:50.652875+0000","last_fullsized":"2023-03-29T14:02:50.652875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4336206,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":13756,"num_write":24,"num_write_kb":6396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.343264+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:49.343264+0000","last_peered":"2023-03-29T14:02:49.343264+0000","last_clean":"2023-03-29T14:02:49.343264+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:49.343264+0000","last_undegraded":"2023-03-29T14:02:49.343264+0000","last_fullsized":"2023-03-29T14:02:49.343264+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.563776+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:50.563776+0000","last_peered":"2023-03-29T14:02:50.563776+0000","last_clean":"2023-03-29T14:02:50.563776+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:50.563776+0000","last_undegraded":"2023-03-29T14:02:50.563776+0000","last_fullsized":"2023-03-29T14:02:50.563776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":14439,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.388989+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:50.388989+0000","last_peered":"2023-03-29T14:02:50.388989+0000","last_clean":"2023-03-29T14:02:50.388989+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:50.388989+0000","last_undegraded":"2023-03-29T14:02:50.388989+0000","last_fullsized":"2023-03-29T14:02:50.388989+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"45'32","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.440333+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:50.440333+0000","last_peered":"2023-03-29T14:02:50.440333+0000","last_clean":"2023-03-29T14:02:50.440333+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:50.440333+0000","last_undegraded":"2023-03-29T14:02:50.440333+0000","last_fullsized":"2023-03-29T14:02:50.440333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17848902,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":23602,"num_write":72,"num_write_kb":22142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.343841+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:49.343841+0000","last_peered":"2023-03-29T14:02:49.343841+0000","last_clean":"2023-03-29T14:02:49.343841+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:49.343841+0000","last_undegraded":"2023-03-29T14:02:49.343841+0000","last_fullsized":"2023-03-29T14:02:49.343841+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":258,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.484234+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:49.484234+0000","last_peered":"2023-03-29T14:02:49.484234+0000","last_clean":"2023-03-29T14:02:49.484234+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:49.484234+0000","last_undegraded":"2023-03-29T14:02:49.484234+0000","last_fullsized":"2023-03-29T14:02:49.484234+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"37'40","reported_seq":133,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.378108+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:50.378108+0000","last_peered":"2023-03-29T14:02:50.378108+0000","last_clean":"2023-03-29T14:02:50.378108+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:50.378108+0000","last_undegraded":"2023-03-29T14:02:50.378108+0000","last_fullsized":"2023-03-29T14:02:50.378108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21411920,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":32183,"num_write":90,"num_write_kb":26440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.334493+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:49.334493+0000","last_peered":"2023-03-29T14:02:49.334493+0000","last_clean":"2023-03-29T14:02:49.334493+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:49.334493+0000","last_undegraded":"2023-03-29T14:02:49.334493+0000","last_fullsized":"2023-03-29T14:02:49.334493+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"45'63","reported_seq":183,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:50.600655+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:50.600655+0000","last_peered":"2023-03-29T14:02:50.600655+0000","last_clean":"2023-03-29T14:02:50.600655+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:50.600655+0000","last_undegraded":"2023-03-29T14:02:50.600655+0000","last_fullsized":"2023-03-29T14:02:50.600655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28605316,"num_objects":16,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":39000,"num_write":140,"num_write_kb":40352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.463476+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:49.463476+0000","last_peered":"2023-03-29T14:02:49.463476+0000","last_clean":"2023-03-29T14:02:49.463476+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:49.463476+0000","last_undegraded":"2023-03-29T14:02:49.463476+0000","last_fullsized":"2023-03-29T14:02:49.463476+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.475188+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:49.475188+0000","last_peered":"2023-03-29T14:02:49.475188+0000","last_clean":"2023-03-29T14:02:49.475188+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:49.475188+0000","last_undegraded":"2023-03-29T14:02:49.475188+0000","last_fullsized":"2023-03-29T14:02:49.475188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":15525,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.459436+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:49.459436+0000","last_peered":"2023-03-29T14:02:49.459436+0000","last_clean":"2023-03-29T14:02:49.459436+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:49.459436+0000","last_undegraded":"2023-03-29T14:02:49.459436+0000","last_fullsized":"2023-03-29T14:02:49.459436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":11149,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.479194+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:49.479194+0000","last_peered":"2023-03-29T14:02:49.479194+0000","last_clean":"2023-03-29T14:02:49.479194+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:49.479194+0000","last_undegraded":"2023-03-29T14:02:49.479194+0000","last_fullsized":"2023-03-29T14:02:49.479194+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.340184+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:49.340184+0000","last_peered":"2023-03-29T14:02:49.340184+0000","last_clean":"2023-03-29T14:02:49.340184+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:49.340184+0000","last_undegraded":"2023-03-29T14:02:49.340184+0000","last_fullsized":"2023-03-29T14:02:49.340184+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"43'27","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-03-29T14:02:49.450371+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:49.450371+0000","last_peered":"2023-03-29T14:02:49.450371+0000","last_clean":"2023-03-29T14:02:49.450371+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:49.450371+0000","last_undegraded":"2023-03-29T14:02:49.450371+0000","last_fullsized":"2023-03-29T14:02:49.450371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630378,"num_objects":8,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":23294,"num_write":60,"num_write_kb":16304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":244875812,"num_objects":146,"num_object_clones":56,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":634,"num_read_kb":292491,"num_write":940,"num_write_kb":270726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":417,"ondisk_log_size":417,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":86616,"kb_used_data":80464,"kb_used_omap":0,"kb_used_meta":6144,"kb_avail":94285224,"statfs":{"total":96636764160,"available":96548069376,"internally_reserved":0,"allocated":82395136,"data_stored":87300690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6291456},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56912,"kb_used_data":50760,"kb_used_omap":0,"kb_used_meta":6144,"kb_avail":94314928,"statfs":{"total":96636764160,"available":96578486272,"internally_reserved":0,"allocated":51978240,"data_stored":53575227,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6291456},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91504,"kb_used_data":84840,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94280336,"statfs":{"total":96636764160,"available":96543064064,"internally_reserved":0,"allocated":86876160,"data_stored":92333897,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":88456,"kb_used_data":81984,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94283384,"statfs":{"total":96636764160,"available":96546185216,"internally_reserved":0,"allocated":83951616,"data_stored":88805443,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:53.323 INFO:tasks.rados.rados.0.smithi017.stdout:222: done (0 left) 2023-03-29T14:02:53.323 INFO:tasks.rados.rados.0.smithi017.stdout:223: write (excl) oid 45 current snap is 17 2023-03-29T14:02:53.323 INFO:tasks.rados.rados.0.smithi017.stdout:223: seq_num 135 ranges {668945=536616,1633422=491363} 2023-03-29T14:02:53.329 INFO:tasks.rados.rados.0.smithi017.stdout:223: writing smithi017107884-45 from 668945 to 1205561 tid 1 2023-03-29T14:02:53.334 INFO:tasks.rados.rados.0.smithi017.stdout:223: writing smithi017107884-45 from 1633422 to 2124785 tid 2 2023-03-29T14:02:53.334 INFO:tasks.rados.rados.0.smithi017.stdout:224: write oid 25 current snap is 17 2023-03-29T14:02:53.335 INFO:tasks.rados.rados.0.smithi017.stdout:224: seq_num 136 ranges {685752=611865,1975551=310201} 2023-03-29T14:02:53.341 INFO:tasks.rados.rados.0.smithi017.stdout:224: writing smithi017107884-25 from 685752 to 1297617 tid 1 2023-03-29T14:02:53.347 INFO:tasks.rados.rados.0.smithi017.stdout:224: writing smithi017107884-25 from 1975551 to 2285752 tid 2 2023-03-29T14:02:53.347 INFO:tasks.rados.rados.0.smithi017.stdout:225: write (excl) oid 19 current snap is 17 2023-03-29T14:02:53.347 INFO:tasks.rados.rados.0.smithi017.stdout:225: seq_num 137 ranges {702559=403467,1750386=495032} 2023-03-29T14:02:53.353 INFO:tasks.rados.rados.0.smithi017.stdout:225: writing smithi017107884-19 from 702559 to 1106026 tid 1 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:225: writing smithi017107884-19 from 1750386 to 2245418 tid 2 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:226: read oid 44 snap -1 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:226: expect (ObjNum 105 snap 9 seq_num 105) 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:227: read oid 32 snap -1 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:227: expect (ObjNum 31 snap 0 seq_num 31) 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:228: read oid 36 snap -1 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:228: expect (ObjNum 127 snap 15 seq_num 127) 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:229: read oid 6 snap -1 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:229: expect (ObjNum 66 snap 3 seq_num 66) 2023-03-29T14:02:53.360 INFO:tasks.rados.rados.0.smithi017.stdout:230: snap_remove snap 15 2023-03-29T14:02:53.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:53.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:53.689+0000 7fe2d9d45700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:53.693+0000 7fe2d9d45700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:54.331 INFO:tasks.rados.rados.0.smithi017.stdout:230: done (7 left) 2023-03-29T14:02:54.331 INFO:tasks.rados.rados.0.smithi017.stdout:231: read oid 31 snap -1 2023-03-29T14:02:54.331 INFO:tasks.rados.rados.0.smithi017.stdout:231: expect (ObjNum 30 snap 0 seq_num 30) 2023-03-29T14:02:54.332 INFO:tasks.rados.rados.0.smithi017.stdout:232: write oid 2 current snap is 17 2023-03-29T14:02:54.332 INFO:tasks.rados.rados.0.smithi017.stdout:232: seq_num 138 ranges {719366=478716,1808868=510498} 2023-03-29T14:02:54.338 INFO:tasks.rados.rados.0.smithi017.stdout:232: writing smithi017107884-2 from 719366 to 1198082 tid 1 2023-03-29T14:02:54.344 INFO:tasks.rados.rados.0.smithi017.stdout:232: writing smithi017107884-2 from 1808868 to 2319366 tid 2 2023-03-29T14:02:54.345 INFO:tasks.rados.rados.0.smithi017.stdout:233: read oid 29 snap -1 2023-03-29T14:02:54.345 INFO:tasks.rados.rados.0.smithi017.stdout:233: expect (ObjNum 90 snap 7 seq_num 90) 2023-03-29T14:02:54.345 INFO:tasks.rados.rados.0.smithi017.stdout:234: write oid 14 current snap is 17 2023-03-29T14:02:54.345 INFO:tasks.rados.rados.0.smithi017.stdout:234: seq_num 139 ranges {736173=553965,1867350=468823} 2023-03-29T14:02:54.351 INFO:tasks.rados.rados.0.smithi017.stdout:234: writing smithi017107884-14 from 736173 to 1290138 tid 1 2023-03-29T14:02:54.357 INFO:tasks.rados.rados.0.smithi017.stdout:234: writing smithi017107884-14 from 1867350 to 2336173 tid 2 2023-03-29T14:02:54.357 INFO:tasks.rados.rados.0.smithi017.stdout:235: delete oid 16 current snap is 17 2023-03-29T14:02:54.357 INFO:tasks.rados.rados.0.smithi017.stdout:229: expect (ObjNum 66 snap 3 seq_num 66) 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: finishing write tid 1 to smithi017107884-25 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: finishing write tid 2 to smithi017107884-25 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: finishing write tid 3 to smithi017107884-25 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: finishing write tid 4 to smithi017107884-25 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: oid 25 updating version 0 to 41 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: oid 25 version 41 is already newer than 41 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: oid 25 version 41 is already newer than 41 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 41 (ObjNum 136 snap 17 seq_num 136) dirty exists 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:224: left oid 25 (ObjNum 136 snap 17 seq_num 136) 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:225: finishing write tid 1 to smithi017107884-19 2023-03-29T14:02:54.521 INFO:tasks.rados.rados.0.smithi017.stdout:223: finishing write tid 1 to smithi017107884-45 2023-03-29T14:02:54.522 INFO:tasks.rados.rados.0.smithi017.stdout:224: done (11 left) 2023-03-29T14:02:54.522 INFO:tasks.rados.rados.0.smithi017.stdout:229: done (10 left) 2023-03-29T14:02:54.522 INFO:tasks.rados.rados.0.smithi017.stdout:235: done (9 left) 2023-03-29T14:02:54.522 INFO:tasks.rados.rados.0.smithi017.stdout:236: read oid 22 snap -1 2023-03-29T14:02:54.522 INFO:tasks.rados.rados.0.smithi017.stdout:236: expect (ObjNum 71 snap 3 seq_num 71) 2023-03-29T14:02:54.522 INFO:tasks.rados.rados.0.smithi017.stdout:237: delete oid 17 current snap is 17 2023-03-29T14:02:54.522 INFO:tasks.rados.rados.0.smithi017.stdout:226: expect (ObjNum 105 snap 9 seq_num 105) 2023-03-29T14:02:54.751 INFO:tasks.rados.rados.0.smithi017.stdout:227: expect (ObjNum 31 snap 0 seq_num 31) 2023-03-29T14:02:54.848 INFO:tasks.rados.rados.0.smithi017.stdout:228: expect (ObjNum 127 snap 15 seq_num 127) 2023-03-29T14:02:54.930 INFO:tasks.rados.rados.0.smithi017.stdout:225: finishing write tid 2 to smithi017107884-19 2023-03-29T14:02:54.930 INFO:tasks.rados.rados.0.smithi017.stdout:226: done (10 left) 2023-03-29T14:02:54.930 INFO:tasks.rados.rados.0.smithi017.stdout:227: done (9 left) 2023-03-29T14:02:54.930 INFO:tasks.rados.rados.0.smithi017.stdout:228: done (8 left) 2023-03-29T14:02:54.930 INFO:tasks.rados.rados.0.smithi017.stdout:237: done (7 left) 2023-03-29T14:02:54.930 INFO:tasks.rados.rados.0.smithi017.stdout:238: snap_remove snap 11 2023-03-29T14:02:55.333 INFO:tasks.rados.rados.0.smithi017.stdout:238: done (7 left) 2023-03-29T14:02:55.334 INFO:tasks.rados.rados.0.smithi017.stdout:239: read oid 25 snap -1 2023-03-29T14:02:55.334 INFO:tasks.rados.rados.0.smithi017.stdout:239: expect (ObjNum 136 snap 17 seq_num 136) 2023-03-29T14:02:55.334 INFO:tasks.rados.rados.0.smithi017.stdout:240: snap_remove snap 16 2023-03-29T14:02:55.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:02:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:55.898+0000 7ffbac9e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:55.902 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:55.903+0000 7ffbac9e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:240: done (8 left) 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:241: read oid 40 snap -1 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:241: expect (ObjNum 122 snap 14 seq_num 122) 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:242: read oid 32 snap -1 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:242: expect (ObjNum 31 snap 0 seq_num 31) 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:243: snap_create 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:223: finishing write tid 2 to smithi017107884-45 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:225: finishing write tid 3 to smithi017107884-19 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:225: finishing write tid 4 to smithi017107884-19 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:225: oid 19 updating version 0 to 28 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:225: oid 19 updating version 28 to 30 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:225: oid 19 version 30 is already newer than 30 2023-03-29T14:02:56.338 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 30 (ObjNum 137 snap 17 seq_num 137) dirty exists 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:225: left oid 19 (ObjNum 137 snap 17 seq_num 137) 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:223: finishing write tid 3 to smithi017107884-45 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:223: finishing write tid 4 to smithi017107884-45 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:223: oid 45 updating version 0 to 40 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:223: oid 45 version 40 is already newer than 40 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:223: oid 45 version 40 is already newer than 37 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 40 (ObjNum 135 snap 17 seq_num 135) dirty exists 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:223: left oid 45 (ObjNum 135 snap 17 seq_num 135) 2023-03-29T14:02:56.339 INFO:tasks.rados.rados.0.smithi017.stdout:231: expect (ObjNum 30 snap 0 seq_num 30) 2023-03-29T14:02:56.436 INFO:tasks.rados.rados.0.smithi017.stdout:233: expect (ObjNum 90 snap 7 seq_num 90) 2023-03-29T14:02:56.619 INFO:tasks.rados.rados.0.smithi017.stdout:232: finishing write tid 1 to smithi017107884-2 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: finishing write tid 1 to smithi017107884-14 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:232: finishing write tid 2 to smithi017107884-2 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: finishing write tid 2 to smithi017107884-14 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: finishing write tid 3 to smithi017107884-14 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: finishing write tid 4 to smithi017107884-14 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: oid 14 updating version 0 to 22 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: oid 14 version 22 is already newer than 22 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: oid 14 version 22 is already newer than 22 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 22 (ObjNum 139 snap 17 seq_num 139) dirty exists 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:234: left oid 14 (ObjNum 139 snap 17 seq_num 139) 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:232: finishing write tid 3 to smithi017107884-2 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:232: finishing write tid 4 to smithi017107884-2 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:232: oid 2 updating version 0 to 72 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:232: oid 2 version 72 is already newer than 72 2023-03-29T14:02:56.620 INFO:tasks.rados.rados.0.smithi017.stdout:232: oid 2 version 72 is already newer than 72 2023-03-29T14:02:56.621 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 72 (ObjNum 138 snap 17 seq_num 138) dirty exists 2023-03-29T14:02:56.621 INFO:tasks.rados.rados.0.smithi017.stdout:232: left oid 2 (ObjNum 138 snap 17 seq_num 138) 2023-03-29T14:02:56.621 INFO:tasks.rados.rados.0.smithi017.stdout:236: expect (ObjNum 71 snap 3 seq_num 71) 2023-03-29T14:02:56.837 INFO:tasks.rados.rados.0.smithi017.stdout:239: expect (ObjNum 136 snap 17 seq_num 136) 2023-03-29T14:02:56.977 INFO:tasks.rados.rados.0.smithi017.stdout:241: expect (ObjNum 122 snap 14 seq_num 122) 2023-03-29T14:02:57.094 INFO:tasks.rados.rados.0.smithi017.stdout:242: expect (ObjNum 31 snap 0 seq_num 31) 2023-03-29T14:02:57.337 INFO:tasks.rados.rados.0.smithi017.stdout:223: done (10 left) 2023-03-29T14:02:57.337 INFO:tasks.rados.rados.0.smithi017.stdout:225: done (9 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:231: done (8 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:232: done (7 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:233: done (6 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:234: done (5 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:236: done (4 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:239: done (3 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:241: done (2 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:242: done (1 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:243: done (0 left) 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:244: write (excl) oid 49 current snap is 18 2023-03-29T14:02:57.338 INFO:tasks.rados.rados.0.smithi017.stdout:244: seq_num 142 ranges {786594=779712,2326443=60151} 2023-03-29T14:02:57.346 INFO:tasks.rados.rados.0.smithi017.stdout:244: writing smithi017107884-49 from 786594 to 1566306 tid 1 2023-03-29T14:02:57.350 INFO:tasks.rados.rados.0.smithi017.stdout:244: writing smithi017107884-49 from 2326443 to 2386594 tid 2 2023-03-29T14:02:57.350 INFO:tasks.rados.rados.0.smithi017.stdout:245: snap_create 2023-03-29T14:02:57.433 INFO:tasks.rados.rados.0.smithi017.stdout:244: finishing write tid 1 to smithi017107884-49 2023-03-29T14:02:57.505 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:02:57.505 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:02:57.505 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:02:57.531 INFO:tasks.rados.rados.0.smithi017.stdout:244: finishing write tid 2 to smithi017107884-49 2023-03-29T14:02:57.532 INFO:tasks.rados.rados.0.smithi017.stdout:244: finishing write tid 3 to smithi017107884-49 2023-03-29T14:02:57.534 INFO:tasks.rados.rados.0.smithi017.stdout:244: finishing write tid 4 to smithi017107884-49 2023-03-29T14:02:57.534 INFO:tasks.rados.rados.0.smithi017.stdout:244: oid 49 updating version 0 to 73 2023-03-29T14:02:57.534 INFO:tasks.rados.rados.0.smithi017.stdout:244: oid 49 updating version 73 to 75 2023-03-29T14:02:57.534 INFO:tasks.rados.rados.0.smithi017.stdout:244: oid 49 version 75 is already newer than 75 2023-03-29T14:02:57.534 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 75 (ObjNum 142 snap 18 seq_num 142) dirty exists 2023-03-29T14:02:57.534 INFO:tasks.rados.rados.0.smithi017.stdout:244: left oid 49 (ObjNum 142 snap 18 seq_num 142) 2023-03-29T14:02:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:57.634+0000 7f08f702c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:57.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:57.641+0000 7f08f702c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:57.842 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:02:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:02:57.855 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-03-29T14:02:57.336399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221081874,"num_objects":170,"num_object_clones":74,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":14,"num_read":849,"num_read_kb":347839,"num_write":1270,"num_write_kb":288338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":342716,"kb_used_data":316828,"kb_used_omap":0,"kb_used_meta":25856,"kb_avail":377144644,"statfs":{"total":386547056640,"available":386196115456,"internally_reserved":0,"allocated":324431872,"data_stored":348179343,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":26476544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24974674,"num_objects":20,"num_object_clones":18,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":73,"num_read_kb":55228,"num_write":56,"num_write_kb":13824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014751"},"pg_stats":[{"pgid":"3.f","version":"49'16","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.537303+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:54.537303+0000","last_peered":"2023-03-29T14:02:54.537303+0000","last_clean":"2023-03-29T14:02:54.537303+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:54.537303+0000","last_undegraded":"2023-03-29T14:02:54.537303+0000","last_fullsized":"2023-03-29T14:02:54.537303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386594,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":50,"num_read_kb":39297,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":49,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.397966+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:54.397966+0000","last_peered":"2023-03-29T14:02:54.397966+0000","last_clean":"2023-03-29T14:02:54.397966+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:54.397966+0000","last_undegraded":"2023-03-29T14:02:54.397966+0000","last_fullsized":"2023-03-29T14:02:54.397966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"49'47","reported_seq":134,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.524838+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:54.524838+0000","last_peered":"2023-03-29T14:02:54.524838+0000","last_clean":"2023-03-29T14:02:54.524838+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:54.524838+0000","last_undegraded":"2023-03-29T14:02:54.524838+0000","last_fullsized":"2023-03-29T14:02:54.524838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15596764,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":40,"num_read_kb":7053,"num_write":98,"num_write_kb":28044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"48'41","reported_seq":130,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:55.369721+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:02:55.369721+0000","last_peered":"2023-03-29T14:02:55.369721+0000","last_clean":"2023-03-29T14:02:55.369721+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:02:55.369721+0000","last_undegraded":"2023-03-29T14:02:55.369721+0000","last_fullsized":"2023-03-29T14:02:55.369721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12941324,"num_objects":18,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":56,"num_read_kb":21315,"num_write":88,"num_write_kb":24426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'40","reported_seq":133,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.381710+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:54.381710+0000","last_peered":"2023-03-29T14:02:54.381710+0000","last_clean":"2023-03-29T14:02:54.381710+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:54.381710+0000","last_undegraded":"2023-03-29T14:02:54.381710+0000","last_fullsized":"2023-03-29T14:02:54.381710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529602,"num_objects":18,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":66,"num_read_kb":34882,"num_write":88,"num_write_kb":25840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.343572+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:54.343572+0000","last_peered":"2023-03-29T14:02:54.343572+0000","last_clean":"2023-03-29T14:02:54.343572+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:54.343572+0000","last_undegraded":"2023-03-29T14:02:54.343572+0000","last_fullsized":"2023-03-29T14:02:54.343572+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'22","reported_seq":91,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.518938+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:54.518938+0000","last_peered":"2023-03-29T14:02:54.518938+0000","last_clean":"2023-03-29T14:02:54.518938+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:54.518938+0000","last_undegraded":"2023-03-29T14:02:54.518938+0000","last_fullsized":"2023-03-29T14:02:54.518938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252204,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":15432,"num_write":50,"num_write_kb":13464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":85,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.354815+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:54.354815+0000","last_peered":"2023-03-29T14:02:54.354815+0000","last_clean":"2023-03-29T14:02:54.354815+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:54.354815+0000","last_undegraded":"2023-03-29T14:02:54.354815+0000","last_fullsized":"2023-03-29T14:02:54.354815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":20934,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.358371+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:54.358371+0000","last_peered":"2023-03-29T14:02:54.358371+0000","last_clean":"2023-03-29T14:02:54.358371+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:54.358371+0000","last_undegraded":"2023-03-29T14:02:54.358371+0000","last_fullsized":"2023-03-29T14:02:54.358371+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"45'13","reported_seq":63,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-03-29T14:02:53.356380+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:02:53.356380+0000","last_peered":"2023-03-29T14:02:53.356380+0000","last_clean":"2023-03-29T14:02:53.356380+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:02:53.356380+0000","last_undegraded":"2023-03-29T14:02:53.356380+0000","last_fullsized":"2023-03-29T14:02:53.356380+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4336206,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":13756,"num_write":24,"num_write_kb":6396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.352712+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:54.352712+0000","last_peered":"2023-03-29T14:02:54.352712+0000","last_clean":"2023-03-29T14:02:54.352712+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:54.352712+0000","last_undegraded":"2023-03-29T14:02:54.352712+0000","last_fullsized":"2023-03-29T14:02:54.352712+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.454473+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:54.454473+0000","last_peered":"2023-03-29T14:02:54.454473+0000","last_clean":"2023-03-29T14:02:54.454473+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:54.454473+0000","last_undegraded":"2023-03-29T14:02:54.454473+0000","last_fullsized":"2023-03-29T14:02:54.454473+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":21266,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-03-29T14:02:53.364466+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:53.364466+0000","last_peered":"2023-03-29T14:02:53.364466+0000","last_clean":"2023-03-29T14:02:53.364466+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:53.364466+0000","last_undegraded":"2023-03-29T14:02:53.364466+0000","last_fullsized":"2023-03-29T14:02:53.364466+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"45'32","reported_seq":116,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.337443+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:02:54.337443+0000","last_peered":"2023-03-29T14:02:54.337443+0000","last_clean":"2023-03-29T14:02:54.337443+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:02:54.337443+0000","last_undegraded":"2023-03-29T14:02:54.337443+0000","last_fullsized":"2023-03-29T14:02:54.337443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17848902,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":29857,"num_write":72,"num_write_kb":22142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.354285+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:54.354285+0000","last_peered":"2023-03-29T14:02:54.354285+0000","last_clean":"2023-03-29T14:02:54.354285+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:54.354285+0000","last_undegraded":"2023-03-29T14:02:54.354285+0000","last_fullsized":"2023-03-29T14:02:54.354285+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":263,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.379961+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:54.379961+0000","last_peered":"2023-03-29T14:02:54.379961+0000","last_clean":"2023-03-29T14:02:54.379961+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:54.379961+0000","last_undegraded":"2023-03-29T14:02:54.379961+0000","last_fullsized":"2023-03-29T14:02:54.379961+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"37'40","reported_seq":141,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.348482+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:54.348482+0000","last_peered":"2023-03-29T14:02:54.348482+0000","last_clean":"2023-03-29T14:02:54.348482+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:54.348482+0000","last_undegraded":"2023-03-29T14:02:54.348482+0000","last_fullsized":"2023-03-29T14:02:54.348482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21411920,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":38262,"num_write":90,"num_write_kb":26440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.350064+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:54.350064+0000","last_peered":"2023-03-29T14:02:54.350064+0000","last_clean":"2023-03-29T14:02:54.350064+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:54.350064+0000","last_undegraded":"2023-03-29T14:02:54.350064+0000","last_fullsized":"2023-03-29T14:02:54.350064+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"49'72","reported_seq":205,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.437259+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:02:54.437259+0000","last_peered":"2023-03-29T14:02:54.437259+0000","last_clean":"2023-03-29T14:02:54.437259+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:02:54.437259+0000","last_undegraded":"2023-03-29T14:02:54.437259+0000","last_fullsized":"2023-03-29T14:02:54.437259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22554928,"num_objects":20,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":91,"num_read_kb":46388,"num_write":156,"num_write_kb":44512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":38,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.368334+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:54.368334+0000","last_peered":"2023-03-29T14:02:54.368334+0000","last_clean":"2023-03-29T14:02:54.368334+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:54.368334+0000","last_undegraded":"2023-03-29T14:02:54.368334+0000","last_fullsized":"2023-03-29T14:02:54.368334+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":113,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.389137+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:54.389137+0000","last_peered":"2023-03-29T14:02:54.389137+0000","last_clean":"2023-03-29T14:02:54.389137+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:54.389137+0000","last_undegraded":"2023-03-29T14:02:54.389137+0000","last_fullsized":"2023-03-29T14:02:54.389137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":24831,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":107,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.372699+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:54.372699+0000","last_peered":"2023-03-29T14:02:54.372699+0000","last_clean":"2023-03-29T14:02:54.372699+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:54.372699+0000","last_undegraded":"2023-03-29T14:02:54.372699+0000","last_fullsized":"2023-03-29T14:02:54.372699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":11149,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.367149+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:54.367149+0000","last_peered":"2023-03-29T14:02:54.367149+0000","last_clean":"2023-03-29T14:02:54.367149+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:54.367149+0000","last_undegraded":"2023-03-29T14:02:54.367149+0000","last_fullsized":"2023-03-29T14:02:54.367149+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-03-29T14:02:54.355665+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:54.355665+0000","last_peered":"2023-03-29T14:02:54.355665+0000","last_clean":"2023-03-29T14:02:54.355665+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:54.355665+0000","last_undegraded":"2023-03-29T14:02:54.355665+0000","last_fullsized":"2023-03-29T14:02:54.355665+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"48'30","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-03-29T14:02:53.565267+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:53.565267+0000","last_peered":"2023-03-29T14:02:53.565267+0000","last_clean":"2023-03-29T14:02:53.565267+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:53.565267+0000","last_undegraded":"2023-03-29T14:02:53.565267+0000","last_fullsized":"2023-03-29T14:02:53.565267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20235496,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":23295,"num_write":68,"num_write_kb":18062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":219901138,"num_objects":166,"num_object_clones":74,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":14,"num_read":707,"num_read_kb":347719,"num_write":996,"num_write_kb":284550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":451,"ondisk_log_size":451,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":91784,"kb_used_data":85504,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":94280056,"statfs":{"total":96636764160,"available":96542777344,"internally_reserved":0,"allocated":87556096,"data_stored":94818181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59860,"kb_used_data":53644,"kb_used_omap":0,"kb_used_meta":6208,"kb_avail":94311980,"statfs":{"total":96636764160,"available":96575467520,"internally_reserved":0,"allocated":54931456,"data_stored":57583814,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6356992},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":97600,"kb_used_data":90808,"kb_used_omap":0,"kb_used_meta":6784,"kb_avail":94274240,"statfs":{"total":96636764160,"available":96536821760,"internally_reserved":0,"allocated":92987392,"data_stored":100620089,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6946816},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705714,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93472,"kb_used_data":86872,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":94278368,"statfs":{"total":96636764160,"available":96541048832,"internally_reserved":0,"allocated":88956928,"data_stored":95157259,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:02:57.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:02:58.106 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:58.102+0000 7f1162d10700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:58.106 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:02:58.106+0000 7f1162d10700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:02:58.343 INFO:tasks.rados.rados.0.smithi017.stdout:244: done (1 left) 2023-03-29T14:02:58.343 INFO:tasks.rados.rados.0.smithi017.stdout:245: done (0 left) 2023-03-29T14:02:58.343 INFO:tasks.rados.rados.0.smithi017.stdout:246: read oid 49 snap -1 2023-03-29T14:02:58.343 INFO:tasks.rados.rados.0.smithi017.stdout:246: expect (ObjNum 142 snap 18 seq_num 142) 2023-03-29T14:02:58.344 INFO:tasks.rados.rados.0.smithi017.stdout:247: write oid 37 current snap is 19 2023-03-29T14:02:58.344 INFO:tasks.rados.rados.0.smithi017.stdout:247: seq_num 143 ranges {403401=454961,1584925=506039} 2023-03-29T14:02:58.348 INFO:tasks.rados.rados.0.smithi017.stdout:247: writing smithi017107884-37 from 403401 to 858362 tid 1 2023-03-29T14:02:58.355 INFO:tasks.rados.rados.0.smithi017.stdout:247: writing smithi017107884-37 from 1584925 to 2090964 tid 2 2023-03-29T14:02:58.355 INFO:tasks.rados.rados.0.smithi017.stdout:248: write oid 48 current snap is 19 2023-03-29T14:02:58.355 INFO:tasks.rados.rados.0.smithi017.stdout:248: seq_num 144 ranges {420208=530210,1527054=766050} 2023-03-29T14:02:58.360 INFO:tasks.rados.rados.0.smithi017.stdout:248: writing smithi017107884-48 from 420208 to 950418 tid 1 2023-03-29T14:02:58.367 INFO:tasks.rados.rados.0.smithi017.stdout:248: writing smithi017107884-48 from 1527054 to 2293104 tid 2 2023-03-29T14:02:58.367 INFO:tasks.rados.rados.0.smithi017.stdout:249: snap_create 2023-03-29T14:02:58.386 INFO:tasks.rados.rados.0.smithi017.stdout:246: expect (ObjNum 142 snap 18 seq_num 142) 2023-03-29T14:02:58.483 INFO:tasks.rados.rados.0.smithi017.stdout:247: finishing write tid 1 to smithi017107884-37 2023-03-29T14:02:58.525 INFO:tasks.rados.rados.0.smithi017.stdout:247: finishing write tid 2 to smithi017107884-37 2023-03-29T14:02:58.526 INFO:tasks.rados.rados.0.smithi017.stdout:247: finishing write tid 3 to smithi017107884-37 2023-03-29T14:02:58.526 INFO:tasks.rados.rados.0.smithi017.stdout:247: finishing write tid 4 to smithi017107884-37 2023-03-29T14:02:58.527 INFO:tasks.rados.rados.0.smithi017.stdout:247: oid 37 updating version 0 to 17 2023-03-29T14:02:58.527 INFO:tasks.rados.rados.0.smithi017.stdout:247: oid 37 version 17 is already newer than 17 2023-03-29T14:02:58.527 INFO:tasks.rados.rados.0.smithi017.stdout:247: oid 37 version 17 is already newer than 17 2023-03-29T14:02:58.527 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 17 (ObjNum 143 snap 19 seq_num 143) dirty exists 2023-03-29T14:02:58.527 INFO:tasks.rados.rados.0.smithi017.stdout:247: left oid 37 (ObjNum 143 snap 19 seq_num 143) 2023-03-29T14:02:58.545 INFO:tasks.rados.rados.0.smithi017.stdout:248: finishing write tid 1 to smithi017107884-48 2023-03-29T14:02:58.545 INFO:tasks.rados.rados.0.smithi017.stdout:248: finishing write tid 2 to smithi017107884-48 2023-03-29T14:02:58.546 INFO:tasks.rados.rados.0.smithi017.stdout:248: finishing write tid 3 to smithi017107884-48 2023-03-29T14:02:58.547 INFO:tasks.rados.rados.0.smithi017.stdout:248: finishing write tid 4 to smithi017107884-48 2023-03-29T14:02:58.547 INFO:tasks.rados.rados.0.smithi017.stdout:248: oid 48 updating version 0 to 50 2023-03-29T14:02:58.547 INFO:tasks.rados.rados.0.smithi017.stdout:248: oid 48 version 50 is already newer than 50 2023-03-29T14:02:58.547 INFO:tasks.rados.rados.0.smithi017.stdout:248: oid 48 version 50 is already newer than 50 2023-03-29T14:02:58.547 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 50 (ObjNum 144 snap 19 seq_num 144) dirty exists 2023-03-29T14:02:58.547 INFO:tasks.rados.rados.0.smithi017.stdout:248: left oid 48 (ObjNum 144 snap 19 seq_num 144) 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:246: done (3 left) 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:247: done (2 left) 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:248: done (1 left) 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:249: done (0 left) 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:250: read oid 44 snap -1 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:250: expect (ObjNum 105 snap 9 seq_num 105) 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:251: write (excl) oid 32 current snap is 20 2023-03-29T14:02:59.345 INFO:tasks.rados.rados.0.smithi017.stdout:251: seq_num 145 ranges {437015=721812,1701889=509708} 2023-03-29T14:02:59.352 INFO:tasks.rados.rados.0.smithi017.stdout:251: writing smithi017107884-32 from 437015 to 1158827 tid 1 2023-03-29T14:02:59.358 INFO:tasks.rados.rados.0.smithi017.stdout:251: writing smithi017107884-32 from 1701889 to 2211597 tid 2 2023-03-29T14:02:59.358 INFO:tasks.rados.rados.0.smithi017.stdout:252: write oid 10 current snap is 20 2023-03-29T14:02:59.359 INFO:tasks.rados.rados.0.smithi017.stdout:252: seq_num 146 ranges {453822=797061,1760371=653366} 2023-03-29T14:02:59.365 INFO:tasks.rados.rados.0.smithi017.stdout:252: writing smithi017107884-10 from 453822 to 1250883 tid 1 2023-03-29T14:02:59.372 INFO:tasks.rados.rados.0.smithi017.stdout:252: writing smithi017107884-10 from 1760371 to 2413737 tid 2 2023-03-29T14:02:59.372 INFO:tasks.rados.rados.0.smithi017.stdout:253: delete oid 46 current snap is 20 2023-03-29T14:02:59.474 INFO:tasks.rados.rados.0.smithi017.stdout:251: finishing write tid 1 to smithi017107884-32 2023-03-29T14:02:59.474 INFO:tasks.rados.rados.0.smithi017.stdout:253: done (3 left) 2023-03-29T14:02:59.474 INFO:tasks.rados.rados.0.smithi017.stdout:254: snap_remove snap 19 2023-03-29T14:03:00.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:00.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:00.308+0000 7f6afbe38700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:00.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:00.312+0000 7f6afbe38700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:00.350 INFO:tasks.rados.rados.0.smithi017.stdout:254: done (3 left) 2023-03-29T14:03:00.351 INFO:tasks.rados.rados.0.smithi017.stdout:255: delete oid 43 current snap is 20 2023-03-29T14:03:00.351 INFO:tasks.rados.rados.0.smithi017.stdout:252: finishing write tid 1 to smithi017107884-10 2023-03-29T14:03:00.351 INFO:tasks.rados.rados.0.smithi017.stdout:252: finishing write tid 2 to smithi017107884-10 2023-03-29T14:03:00.351 INFO:tasks.rados.rados.0.smithi017.stdout:250: expect (ObjNum 105 snap 9 seq_num 105) 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:252: finishing write tid 3 to smithi017107884-10 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:252: finishing write tid 4 to smithi017107884-10 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:252: oid 10 updating version 0 to 53 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:252: oid 10 version 53 is already newer than 53 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:252: oid 10 version 53 is already newer than 53 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 53 (ObjNum 146 snap 20 seq_num 146) dirty exists 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:252: left oid 10 (ObjNum 146 snap 20 seq_num 146) 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:251: finishing write tid 2 to smithi017107884-32 2023-03-29T14:03:00.584 INFO:tasks.rados.rados.0.smithi017.stdout:251: finishing write tid 3 to smithi017107884-32 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:251: finishing write tid 4 to smithi017107884-32 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:251: oid 32 updating version 0 to 76 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:251: oid 32 updating version 76 to 79 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:251: oid 32 version 79 is already newer than 79 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 79 (ObjNum 145 snap 20 seq_num 145) dirty exists 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:251: left oid 32 (ObjNum 145 snap 20 seq_num 145) 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:250: done (3 left) 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:251: done (2 left) 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:252: done (1 left) 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:255: done (0 left) 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:256: read oid 25 snap 18 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:256: expect (ObjNum 136 snap 17 seq_num 136) 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:257: write (excl) oid 30 current snap is 20 2023-03-29T14:03:00.585 INFO:tasks.rados.rados.0.smithi017.stdout:257: seq_num 149 ranges {504243=622808,1819464=517046} 2023-03-29T14:03:00.590 INFO:tasks.rados.rados.0.smithi017.stdout:257: writing smithi017107884-30 from 504243 to 1127051 tid 1 2023-03-29T14:03:00.597 INFO:tasks.rados.rados.0.smithi017.stdout:257: writing smithi017107884-30 from 1819464 to 2336510 tid 2 2023-03-29T14:03:00.597 INFO:tasks.rados.rados.0.smithi017.stdout:258: write oid 3 current snap is 20 2023-03-29T14:03:00.597 INFO:tasks.rados.rados.0.smithi017.stdout:258: seq_num 150 ranges {521050=698057,1877946=643104} 2023-03-29T14:03:00.603 INFO:tasks.rados.rados.0.smithi017.stdout:258: writing smithi017107884-3 from 521050 to 1219107 tid 1 2023-03-29T14:03:00.611 INFO:tasks.rados.rados.0.smithi017.stdout:258: writing smithi017107884-3 from 1877946 to 2521050 tid 2 2023-03-29T14:03:00.611 INFO:tasks.rados.rados.0.smithi017.stdout:259: snap_create 2023-03-29T14:03:00.640 INFO:tasks.rados.rados.0.smithi017.stdout:256: expect (ObjNum 136 snap 17 seq_num 136) 2023-03-29T14:03:00.782 INFO:tasks.rados.rados.0.smithi017.stdout:257: finishing write tid 1 to smithi017107884-30 2023-03-29T14:03:00.782 INFO:tasks.rados.rados.0.smithi017.stdout:258: finishing write tid 1 to smithi017107884-3 2023-03-29T14:03:00.782 INFO:tasks.rados.rados.0.smithi017.stdout:258: finishing write tid 2 to smithi017107884-3 2023-03-29T14:03:00.782 INFO:tasks.rados.rados.0.smithi017.stdout:258: finishing write tid 3 to smithi017107884-3 2023-03-29T14:03:00.782 INFO:tasks.rados.rados.0.smithi017.stdout:258: finishing write tid 4 to smithi017107884-3 2023-03-29T14:03:00.782 INFO:tasks.rados.rados.0.smithi017.stdout:258: oid 3 updating version 0 to 83 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:258: oid 3 version 83 is already newer than 83 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:258: oid 3 version 83 is already newer than 83 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 83 (ObjNum 150 snap 20 seq_num 150) dirty exists 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:258: left oid 3 (ObjNum 150 snap 20 seq_num 150) 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:257: finishing write tid 2 to smithi017107884-30 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:257: finishing write tid 3 to smithi017107884-30 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:257: finishing write tid 4 to smithi017107884-30 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:257: oid 30 updating version 0 to 18 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:257: oid 30 updating version 18 to 20 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:257: oid 30 version 20 is already newer than 20 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 20 (ObjNum 149 snap 20 seq_num 149) dirty exists 2023-03-29T14:03:00.783 INFO:tasks.rados.rados.0.smithi017.stdout:257: left oid 30 (ObjNum 149 snap 20 seq_num 149) 2023-03-29T14:03:01.354 INFO:tasks.rados.rados.0.smithi017.stdout:256: done (3 left) 2023-03-29T14:03:01.354 INFO:tasks.rados.rados.0.smithi017.stdout:257: done (2 left) 2023-03-29T14:03:01.355 INFO:tasks.rados.rados.0.smithi017.stdout:258: done (1 left) 2023-03-29T14:03:01.355 INFO:tasks.rados.rados.0.smithi017.stdout:259: done (0 left) 2023-03-29T14:03:01.355 INFO:tasks.rados.rados.0.smithi017.stdout:260: write (excl) oid 45 current snap is 21 2023-03-29T14:03:01.355 INFO:tasks.rados.rados.0.smithi017.stdout:260: seq_num 151 ranges {537857=773306,1936428=520715} 2023-03-29T14:03:01.361 INFO:tasks.rados.rados.0.smithi017.stdout:260: writing smithi017107884-45 from 537857 to 1311163 tid 1 2023-03-29T14:03:01.368 INFO:tasks.rados.rados.0.smithi017.stdout:260: writing smithi017107884-45 from 1936428 to 2457143 tid 2 2023-03-29T14:03:01.368 INFO:tasks.rados.rados.0.smithi017.stdout:261: snap_remove snap 17 2023-03-29T14:03:02.359 INFO:tasks.rados.rados.0.smithi017.stdout:261: done (1 left) 2023-03-29T14:03:02.359 INFO:tasks.rados.rados.0.smithi017.stdout:262: write (excl) oid 9 current snap is 21 2023-03-29T14:03:02.359 INFO:tasks.rados.rados.0.smithi017.stdout:260: finishing write tid 1 to smithi017107884-45 2023-03-29T14:03:02.359 INFO:tasks.rados.rados.0.smithi017.stdout:262: seq_num 152 ranges {554664=448555,1478557=664373} 2023-03-29T14:03:02.364 INFO:tasks.rados.rados.0.smithi017.stdout:262: writing smithi017107884-9 from 554664 to 1003219 tid 1 2023-03-29T14:03:02.371 INFO:tasks.rados.rados.0.smithi017.stdout:262: writing smithi017107884-9 from 1478557 to 2142930 tid 2 2023-03-29T14:03:02.371 INFO:tasks.rados.rados.0.smithi017.stdout:263: write (excl) oid 20 current snap is 21 2023-03-29T14:03:02.371 INFO:tasks.rados.rados.0.smithi017.stdout:263: seq_num 153 ranges {571471=640157,1653392=524384} 2023-03-29T14:03:02.378 INFO:tasks.rados.rados.0.smithi017.stdout:263: writing smithi017107884-20 from 571471 to 1211628 tid 1 2023-03-29T14:03:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:263: writing smithi017107884-20 from 1653392 to 2177776 tid 2 2023-03-29T14:03:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:264: snap_create 2023-03-29T14:03:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:260: finishing write tid 2 to smithi017107884-45 2023-03-29T14:03:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:260: finishing write tid 3 to smithi017107884-45 2023-03-29T14:03:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:260: finishing write tid 4 to smithi017107884-45 2023-03-29T14:03:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:260: oid 45 updating version 0 to 45 2023-03-29T14:03:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:260: oid 45 version 45 is already newer than 42 2023-03-29T14:03:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:260: oid 45 version 45 is already newer than 45 2023-03-29T14:03:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 45 (ObjNum 151 snap 21 seq_num 151) dirty exists 2023-03-29T14:03:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:260: left oid 45 (ObjNum 151 snap 21 seq_num 151) 2023-03-29T14:03:02.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:02.472 INFO:tasks.rados.rados.0.smithi017.stdout:262: finishing write tid 1 to smithi017107884-9 2023-03-29T14:03:02.490 INFO:tasks.rados.rados.0.smithi017.stdout:263: finishing write tid 1 to smithi017107884-20 2023-03-29T14:03:02.505 INFO:tasks.rados.rados.0.smithi017.stdout:262: finishing write tid 2 to smithi017107884-9 2023-03-29T14:03:02.521 INFO:tasks.rados.rados.0.smithi017.stdout:262: finishing write tid 3 to smithi017107884-9 2023-03-29T14:03:02.522 INFO:tasks.rados.rados.0.smithi017.stdout:262: finishing write tid 4 to smithi017107884-9 2023-03-29T14:03:02.522 INFO:tasks.rados.rados.0.smithi017.stdout:262: oid 9 updating version 0 to 45 2023-03-29T14:03:02.522 INFO:tasks.rados.rados.0.smithi017.stdout:262: oid 9 version 45 is already newer than 45 2023-03-29T14:03:02.522 INFO:tasks.rados.rados.0.smithi017.stdout:262: oid 9 version 45 is already newer than 42 2023-03-29T14:03:02.522 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 45 (ObjNum 152 snap 21 seq_num 152) dirty exists 2023-03-29T14:03:02.522 INFO:tasks.rados.rados.0.smithi017.stdout:262: left oid 9 (ObjNum 152 snap 21 seq_num 152) 2023-03-29T14:03:02.522 INFO:tasks.rados.rados.0.smithi017.stdout:263: finishing write tid 2 to smithi017107884-20 2023-03-29T14:03:02.523 INFO:tasks.rados.rados.0.smithi017.stdout:263: finishing write tid 3 to smithi017107884-20 2023-03-29T14:03:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:263: finishing write tid 4 to smithi017107884-20 2023-03-29T14:03:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:263: oid 20 updating version 0 to 46 2023-03-29T14:03:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:263: oid 20 updating version 46 to 49 2023-03-29T14:03:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:263: oid 20 version 49 is already newer than 49 2023-03-29T14:03:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 49 (ObjNum 153 snap 21 seq_num 153) dirty exists 2023-03-29T14:03:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:263: left oid 20 (ObjNum 153 snap 21 seq_num 153) 2023-03-29T14:03:02.540 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:02.539+0000 7fac90cfa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:02.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:02.543+0000 7fac90cfa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:02.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:02.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:02.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:02.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:02.994+0000 7f5be27d7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:03.000 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:03.000+0000 7f5be27d7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:03.203 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:03.206 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:03.218 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-03-29T14:03:02.357947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235031816,"num_objects":182,"num_object_clones":82,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":898,"num_read_kb":376868,"num_write":1326,"num_write_kb":304486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":572,"ondisk_log_size":572,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":367624,"kb_used_data":341288,"kb_used_omap":0,"kb_used_meta":26304,"kb_avail":377119736,"statfs":{"total":386547056640,"available":386170609664,"internally_reserved":0,"allocated":349478912,"data_stored":376668289,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":26935296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13949942,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":49,"num_read_kb":29029,"num_write":56,"num_write_kb":16148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015770"},"pg_stats":[{"pgid":"3.f","version":"49'16","reported_seq":87,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:58.434311+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:02:58.434311+0000","last_peered":"2023-03-29T14:02:58.434311+0000","last_clean":"2023-03-29T14:02:58.434311+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:02:58.434311+0000","last_undegraded":"2023-03-29T14:02:58.434311+0000","last_fullsized":"2023-03-29T14:02:58.434311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386594,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":50,"num_read_kb":39297,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":54,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.420078+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:02:59.420078+0000","last_peered":"2023-03-29T14:02:59.420078+0000","last_clean":"2023-03-29T14:02:59.420078+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:02:59.420078+0000","last_undegraded":"2023-03-29T14:02:59.420078+0000","last_fullsized":"2023-03-29T14:02:59.420078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"54'53","reported_seq":153,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.500993+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:02:59.500993+0000","last_peered":"2023-03-29T14:02:59.500993+0000","last_clean":"2023-03-29T14:02:59.500993+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:02:59.500993+0000","last_undegraded":"2023-03-29T14:02:59.500993+0000","last_fullsized":"2023-03-29T14:02:59.500993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25344824,"num_objects":16,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":42,"num_read_kb":7055,"num_write":114,"num_write_kb":33414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"48'41","reported_seq":141,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:03:00.636718+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:00.636718+0000","last_peered":"2023-03-29T14:03:00.636718+0000","last_clean":"2023-03-29T14:03:00.636718+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:00.636718+0000","last_undegraded":"2023-03-29T14:03:00.636718+0000","last_fullsized":"2023-03-29T14:03:00.636718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12941324,"num_objects":18,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":70,"num_read_kb":31795,"num_write":88,"num_write_kb":24426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'40","reported_seq":139,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.465405+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:02:59.465405+0000","last_peered":"2023-03-29T14:02:59.465405+0000","last_clean":"2023-03-29T14:02:59.465405+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:02:59.465405+0000","last_undegraded":"2023-03-29T14:02:59.465405+0000","last_fullsized":"2023-03-29T14:02:59.465405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529602,"num_objects":18,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":67,"num_read_kb":34882,"num_write":88,"num_write_kb":25840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":41,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.386549+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:02:59.386549+0000","last_peered":"2023-03-29T14:02:59.386549+0000","last_clean":"2023-03-29T14:02:59.386549+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:02:59.386549+0000","last_undegraded":"2023-03-29T14:02:59.386549+0000","last_fullsized":"2023-03-29T14:02:59.386549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'22","reported_seq":96,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.391165+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:02:59.391165+0000","last_peered":"2023-03-29T14:02:59.391165+0000","last_clean":"2023-03-29T14:02:59.391165+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:02:59.391165+0000","last_undegraded":"2023-03-29T14:02:59.391165+0000","last_fullsized":"2023-03-29T14:02:59.391165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252204,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":15432,"num_write":50,"num_write_kb":13464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":90,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.390529+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:02:59.390529+0000","last_peered":"2023-03-29T14:02:59.390529+0000","last_clean":"2023-03-29T14:02:59.390529+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:02:59.390529+0000","last_undegraded":"2023-03-29T14:02:59.390529+0000","last_fullsized":"2023-03-29T14:02:59.390529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":20934,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.392700+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:02:59.392700+0000","last_peered":"2023-03-29T14:02:59.392700+0000","last_clean":"2023-03-29T14:02:59.392700+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:02:59.392700+0000","last_undegraded":"2023-03-29T14:02:59.392700+0000","last_fullsized":"2023-03-29T14:02:59.392700+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'20","reported_seq":80,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:03:00.682732+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:00.682732+0000","last_peered":"2023-03-29T14:03:00.682732+0000","last_clean":"2023-03-29T14:03:00.682732+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:00.682732+0000","last_undegraded":"2023-03-29T14:03:00.682732+0000","last_fullsized":"2023-03-29T14:03:00.682732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9815288,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13758,"num_write":40,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.389838+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:02:59.389838+0000","last_peered":"2023-03-29T14:02:59.389838+0000","last_clean":"2023-03-29T14:02:59.389838+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:02:59.389838+0000","last_undegraded":"2023-03-29T14:02:59.389838+0000","last_fullsized":"2023-03-29T14:02:59.389838+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":74,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.368788+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:02:59.368788+0000","last_peered":"2023-03-29T14:02:59.368788+0000","last_clean":"2023-03-29T14:02:59.368788+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:02:59.368788+0000","last_undegraded":"2023-03-29T14:02:59.368788+0000","last_fullsized":"2023-03-29T14:02:59.368788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":21266,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:58.399098+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:02:58.399098+0000","last_peered":"2023-03-29T14:02:58.399098+0000","last_clean":"2023-03-29T14:02:58.399098+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:02:58.399098+0000","last_undegraded":"2023-03-29T14:02:58.399098+0000","last_fullsized":"2023-03-29T14:02:58.399098+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"45'32","reported_seq":123,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:03:00.391811+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:00.391811+0000","last_peered":"2023-03-29T14:03:00.391811+0000","last_clean":"2023-03-29T14:03:00.391811+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:00.391811+0000","last_undegraded":"2023-03-29T14:03:00.391811+0000","last_fullsized":"2023-03-29T14:03:00.391811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10924484,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":29857,"num_write":72,"num_write_kb":22142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.387663+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:02:59.387663+0000","last_peered":"2023-03-29T14:02:59.387663+0000","last_clean":"2023-03-29T14:02:59.387663+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:02:59.387663+0000","last_undegraded":"2023-03-29T14:02:59.387663+0000","last_fullsized":"2023-03-29T14:02:59.387663+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.428166+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:02:59.428166+0000","last_peered":"2023-03-29T14:02:59.428166+0000","last_clean":"2023-03-29T14:02:59.428166+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:02:59.428166+0000","last_undegraded":"2023-03-29T14:02:59.428166+0000","last_fullsized":"2023-03-29T14:02:59.428166+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"37'40","reported_seq":146,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.392587+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:02:59.392587+0000","last_peered":"2023-03-29T14:02:59.392587+0000","last_clean":"2023-03-29T14:02:59.392587+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:02:59.392587+0000","last_undegraded":"2023-03-29T14:02:59.392587+0000","last_fullsized":"2023-03-29T14:02:59.392587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21411920,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":38262,"num_write":90,"num_write_kb":26440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.388403+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:02:59.388403+0000","last_peered":"2023-03-29T14:02:59.388403+0000","last_clean":"2023-03-29T14:02:59.388403+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:02:59.388403+0000","last_undegraded":"2023-03-29T14:02:59.388403+0000","last_fullsized":"2023-03-29T14:02:59.388403+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"55'83","reported_seq":237,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:03:00.660463+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:00.660463+0000","last_peered":"2023-03-29T14:03:00.660463+0000","last_clean":"2023-03-29T14:03:00.660463+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:00.660463+0000","last_undegraded":"2023-03-29T14:03:00.660463+0000","last_fullsized":"2023-03-29T14:03:00.660463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28202146,"num_objects":24,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":55627,"num_write":180,"num_write_kb":51182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.380721+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:02:59.380721+0000","last_peered":"2023-03-29T14:02:59.380721+0000","last_clean":"2023-03-29T14:02:59.380721+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:02:59.380721+0000","last_undegraded":"2023-03-29T14:02:59.380721+0000","last_fullsized":"2023-03-29T14:02:59.380721+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":121,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.488831+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:02:59.488831+0000","last_peered":"2023-03-29T14:02:59.488831+0000","last_clean":"2023-03-29T14:02:59.488831+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:02:59.488831+0000","last_undegraded":"2023-03-29T14:02:59.488831+0000","last_fullsized":"2023-03-29T14:02:59.488831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":112,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.430020+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:02:59.430020+0000","last_peered":"2023-03-29T14:02:59.430020+0000","last_clean":"2023-03-29T14:02:59.430020+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:02:59.430020+0000","last_undegraded":"2023-03-29T14:02:59.430020+0000","last_fullsized":"2023-03-29T14:02:59.430020+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":11149,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.388631+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:02:59.388631+0000","last_peered":"2023-03-29T14:02:59.388631+0000","last_clean":"2023-03-29T14:02:59.388631+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:02:59.388631+0000","last_undegraded":"2023-03-29T14:02:59.388631+0000","last_fullsized":"2023-03-29T14:02:59.388631+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.380305+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:02:59.380305+0000","last_peered":"2023-03-29T14:02:59.380305+0000","last_clean":"2023-03-29T14:02:59.380305+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:02:59.380305+0000","last_undegraded":"2023-03-29T14:02:59.380305+0000","last_fullsized":"2023-03-29T14:02:59.380305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"48'30","reported_seq":121,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-03-29T14:02:59.404637+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:02:59.404637+0000","last_peered":"2023-03-29T14:02:59.404637+0000","last_clean":"2023-03-29T14:02:59.404637+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:02:59.404637+0000","last_undegraded":"2023-03-29T14:02:59.404637+0000","last_fullsized":"2023-03-29T14:02:59.404637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20235496,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":23295,"num_write":68,"num_write_kb":18062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":233851080,"num_objects":178,"num_object_clones":82,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":756,"num_read_kb":376748,"num_write":1052,"num_write_kb":300698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":475,"ondisk_log_size":475,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":100056,"kb_used_data":93648,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94271784,"statfs":{"total":96636764160,"available":96534306816,"internally_reserved":0,"allocated":95895552,"data_stored":104307106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65432,"kb_used_data":59088,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":94306408,"statfs":{"total":96636764160,"available":96569761792,"internally_reserved":0,"allocated":60506112,"data_stored":64329229,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705719,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105876,"kb_used_data":98956,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94265964,"statfs":{"total":96636764160,"available":96528347136,"internally_reserved":0,"allocated":101330944,"data_stored":110112191,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705719,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96260,"kb_used_data":89596,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94275580,"statfs":{"total":96636764160,"available":96538193920,"internally_reserved":0,"allocated":91746304,"data_stored":97919763,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:03.362 INFO:tasks.rados.rados.0.smithi017.stdout:260: done (3 left) 2023-03-29T14:03:03.362 INFO:tasks.rados.rados.0.smithi017.stdout:262: done (2 left) 2023-03-29T14:03:03.362 INFO:tasks.rados.rados.0.smithi017.stdout:263: done (1 left) 2023-03-29T14:03:03.362 INFO:tasks.rados.rados.0.smithi017.stdout:264: done (0 left) 2023-03-29T14:03:03.362 INFO:tasks.rados.rados.0.smithi017.stdout:265: snap_create 2023-03-29T14:03:04.366 INFO:tasks.rados.rados.0.smithi017.stdout:265: done (0 left) 2023-03-29T14:03:04.366 INFO:tasks.rados.rados.0.smithi017.stdout:266: snap_remove snap 22 2023-03-29T14:03:04.612 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:04.780 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:04.779+0000 7f082bc03700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:04.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:04.784+0000 7f082a9a1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:05.366 INFO:tasks.rados.rados.0.smithi017.stdout:266: done (0 left) 2023-03-29T14:03:05.367 INFO:tasks.rados.rados.0.smithi017.stdout:267: write (excl) oid 9 current snap is 23 2023-03-29T14:03:05.367 INFO:tasks.rados.rados.0.smithi017.stdout:267: seq_num 154 ranges {588278=715406,1711874=668042} 2023-03-29T14:03:05.374 INFO:tasks.rados.rados.0.smithi017.stdout:267: writing smithi017107884-9 from 588278 to 1303684 tid 1 2023-03-29T14:03:05.380 INFO:tasks.rados.rados.0.smithi017.stdout:267: writing smithi017107884-9 from 1711874 to 2379916 tid 2 2023-03-29T14:03:05.380 INFO:tasks.rados.rados.0.smithi017.stdout:268: snap_create 2023-03-29T14:03:05.550 INFO:tasks.rados.rados.0.smithi017.stdout:267: finishing write tid 1 to smithi017107884-9 2023-03-29T14:03:05.605 INFO:tasks.rados.rados.0.smithi017.stdout:267: finishing write tid 2 to smithi017107884-9 2023-03-29T14:03:05.605 INFO:tasks.rados.rados.0.smithi017.stdout:267: finishing write tid 3 to smithi017107884-9 2023-03-29T14:03:05.606 INFO:tasks.rados.rados.0.smithi017.stdout:267: finishing write tid 4 to smithi017107884-9 2023-03-29T14:03:05.606 INFO:tasks.rados.rados.0.smithi017.stdout:267: oid 9 updating version 0 to 49 2023-03-29T14:03:05.606 INFO:tasks.rados.rados.0.smithi017.stdout:267: oid 9 version 49 is already newer than 46 2023-03-29T14:03:05.607 INFO:tasks.rados.rados.0.smithi017.stdout:267: oid 9 version 49 is already newer than 49 2023-03-29T14:03:05.607 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 49 (ObjNum 154 snap 23 seq_num 154) dirty exists 2023-03-29T14:03:05.607 INFO:tasks.rados.rados.0.smithi017.stdout:267: left oid 9 (ObjNum 154 snap 23 seq_num 154) 2023-03-29T14:03:06.373 INFO:tasks.rados.rados.0.smithi017.stdout:267: done (1 left) 2023-03-29T14:03:06.373 INFO:tasks.rados.rados.0.smithi017.stdout:268: done (0 left) 2023-03-29T14:03:06.373 INFO:tasks.rados.rados.0.smithi017.stdout:269: read oid 33 snap -1 2023-03-29T14:03:06.373 INFO:tasks.rados.rados.0.smithi017.stdout:269: expect (ObjNum 124 snap 15 seq_num 124) 2023-03-29T14:03:06.373 INFO:tasks.rados.rados.0.smithi017.stdout:270: delete oid 39 current snap is 24 2023-03-29T14:03:06.473 INFO:tasks.rados.rados.0.smithi017.stdout:269: expect (ObjNum 124 snap 15 seq_num 124) 2023-03-29T14:03:06.584 INFO:tasks.rados.rados.0.smithi017.stdout:269: done (1 left) 2023-03-29T14:03:06.584 INFO:tasks.rados.rados.0.smithi017.stdout:270: done (0 left) 2023-03-29T14:03:06.584 INFO:tasks.rados.rados.0.smithi017.stdout:271: snap_remove snap 21 2023-03-29T14:03:06.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:06.986+0000 7f8226d56700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:06.991+0000 7f8226d56700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:07.376 INFO:tasks.rados.rados.0.smithi017.stdout:271: done (0 left) 2023-03-29T14:03:07.377 INFO:tasks.rados.rados.0.smithi017.stdout:272: write (excl) oid 2 current snap is 24 2023-03-29T14:03:07.377 INFO:tasks.rados.rados.0.smithi017.stdout:272: seq_num 156 ranges {621892=465904,1712485=671711} 2023-03-29T14:03:07.382 INFO:tasks.rados.rados.0.smithi017.stdout:272: writing smithi017107884-2 from 621892 to 1087796 tid 1 2023-03-29T14:03:07.389 INFO:tasks.rados.rados.0.smithi017.stdout:272: writing smithi017107884-2 from 1712485 to 2384196 tid 2 2023-03-29T14:03:07.389 INFO:tasks.rados.rados.0.smithi017.stdout:273: delete oid 8 current snap is 24 2023-03-29T14:03:07.474 INFO:tasks.rados.rados.0.smithi017.stdout:272: finishing write tid 1 to smithi017107884-2 2023-03-29T14:03:07.482 INFO:tasks.rados.rados.0.smithi017.stdout:273: done (1 left) 2023-03-29T14:03:07.482 INFO:tasks.rados.rados.0.smithi017.stdout:274: delete oid 42 current snap is 24 2023-03-29T14:03:07.531 INFO:tasks.rados.rados.0.smithi017.stdout:274: done (1 left) 2023-03-29T14:03:07.532 INFO:tasks.rados.rados.0.smithi017.stdout:275: read oid 27 snap -1 2023-03-29T14:03:07.532 INFO:tasks.rados.rados.0.smithi017.stdout:275: expect (ObjNum 74 snap 3 seq_num 74) 2023-03-29T14:03:07.532 INFO:tasks.rados.rados.0.smithi017.stdout:276: snap_remove snap 20 2023-03-29T14:03:08.222 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:08.222 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:08.223 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:08.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:08.357+0000 7f728fc14700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:08.364+0000 7f728fc14700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:08.383 INFO:tasks.rados.rados.0.smithi017.stdout:276: done (2 left) 2023-03-29T14:03:08.383 INFO:tasks.rados.rados.0.smithi017.stdout:277: delete oid 32 current snap is 24 2023-03-29T14:03:08.383 INFO:tasks.rados.rados.0.smithi017.stdout:272: finishing write tid 2 to smithi017107884-2 2023-03-29T14:03:08.383 INFO:tasks.rados.rados.0.smithi017.stdout:272: finishing write tid 3 to smithi017107884-2 2023-03-29T14:03:08.383 INFO:tasks.rados.rados.0.smithi017.stdout:272: finishing write tid 4 to smithi017107884-2 2023-03-29T14:03:08.384 INFO:tasks.rados.rados.0.smithi017.stdout:272: oid 2 updating version 0 to 87 2023-03-29T14:03:08.384 INFO:tasks.rados.rados.0.smithi017.stdout:272: oid 2 version 87 is already newer than 84 2023-03-29T14:03:08.384 INFO:tasks.rados.rados.0.smithi017.stdout:272: oid 2 version 87 is already newer than 87 2023-03-29T14:03:08.384 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 87 (ObjNum 156 snap 24 seq_num 156) dirty exists 2023-03-29T14:03:08.384 INFO:tasks.rados.rados.0.smithi017.stdout:272: left oid 2 (ObjNum 156 snap 24 seq_num 156) 2023-03-29T14:03:08.384 INFO:tasks.rados.rados.0.smithi017.stdout:275: expect (ObjNum 74 snap 3 seq_num 74) 2023-03-29T14:03:08.528 INFO:tasks.rados.rados.0.smithi017.stdout:272: done (2 left) 2023-03-29T14:03:08.529 INFO:tasks.rados.rados.0.smithi017.stdout:275: done (1 left) 2023-03-29T14:03:08.529 INFO:tasks.rados.rados.0.smithi017.stdout:277: done (0 left) 2023-03-29T14:03:08.529 INFO:tasks.rados.rados.0.smithi017.stdout:278: write (excl) oid 37 current snap is 24 2023-03-29T14:03:08.529 INFO:tasks.rados.rados.0.smithi017.stdout:278: seq_num 160 ranges {689120=483253,1946413=679049} 2023-03-29T14:03:08.535 INFO:tasks.rados.rados.0.smithi017.stdout:278: writing smithi017107884-37 from 689120 to 1172373 tid 1 2023-03-29T14:03:08.543 INFO:tasks.rados.rados.0.smithi017.stdout:278: writing smithi017107884-37 from 1946413 to 2625462 tid 2 2023-03-29T14:03:08.543 INFO:tasks.rados.rados.0.smithi017.stdout:279: write (excl) oid 50 current snap is 24 2023-03-29T14:03:08.543 INFO:tasks.rados.rados.0.smithi017.stdout:279: seq_num 161 ranges {705927=558502,2004895=422707} 2023-03-29T14:03:08.550 INFO:tasks.rados.rados.0.smithi017.stdout:279: writing smithi017107884-50 from 705927 to 1264429 tid 1 2023-03-29T14:03:08.556 INFO:tasks.rados.rados.0.smithi017.stdout:279: writing smithi017107884-50 from 2004895 to 2427602 tid 2 2023-03-29T14:03:08.556 INFO:tasks.rados.rados.0.smithi017.stdout:280: snap_remove snap 18 2023-03-29T14:03:08.577 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:08.579 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:08.591 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-03-29T14:03:08.380380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233788230,"num_objects":190,"num_object_clones":90,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":910,"num_read_kb":376872,"num_write":1358,"num_write_kb":314174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":591,"ondisk_log_size":591,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":382528,"kb_used_data":356000,"kb_used_omap":0,"kb_used_meta":26496,"kb_avail":377104832,"statfs":{"total":386547056640,"available":386155347968,"internally_reserved":0,"allocated":364544000,"data_stored":395302912,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":27131904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1243586,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":32,"num_write_kb":9688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015490"},"pg_stats":[{"pgid":"3.f","version":"49'16","reported_seq":91,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.407695+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:05.407695+0000","last_peered":"2023-03-29T14:03:05.407695+0000","last_clean":"2023-03-29T14:03:05.407695+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:05.407695+0000","last_undegraded":"2023-03-29T14:03:05.407695+0000","last_fullsized":"2023-03-29T14:03:05.407695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386594,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":50,"num_read_kb":39297,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":58,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.385336+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:04.385336+0000","last_peered":"2023-03-29T14:03:04.385336+0000","last_clean":"2023-03-29T14:03:04.385336+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:04.385336+0000","last_undegraded":"2023-03-29T14:03:04.385336+0000","last_fullsized":"2023-03-29T14:03:04.385336+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"54'53","reported_seq":157,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.229395+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:05.229395+0000","last_peered":"2023-03-29T14:03:05.229395+0000","last_clean":"2023-03-29T14:03:05.229395+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:05.229395+0000","last_undegraded":"2023-03-29T14:03:05.229395+0000","last_fullsized":"2023-03-29T14:03:05.229395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25344824,"num_objects":16,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":42,"num_read_kb":7055,"num_write":114,"num_write_kb":33414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"60'49","reported_seq":159,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.555299+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:05.555299+0000","last_peered":"2023-03-29T14:03:05.555299+0000","last_clean":"2023-03-29T14:03:05.555299+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:05.555299+0000","last_undegraded":"2023-03-29T14:03:05.555299+0000","last_fullsized":"2023-03-29T14:03:05.555299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13848968,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":31797,"num_write":104,"num_write_kb":29306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":157,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.412668+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:04.412668+0000","last_peered":"2023-03-29T14:03:04.412668+0000","last_clean":"2023-03-29T14:03:04.412668+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:04.412668+0000","last_undegraded":"2023-03-29T14:03:04.412668+0000","last_fullsized":"2023-03-29T14:03:04.412668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":73,"num_read_kb":34884,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.249350+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:05.249350+0000","last_peered":"2023-03-29T14:03:05.249350+0000","last_clean":"2023-03-29T14:03:05.249350+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:05.249350+0000","last_undegraded":"2023-03-29T14:03:05.249350+0000","last_fullsized":"2023-03-29T14:03:05.249350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'22","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.246660+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:05.246660+0000","last_peered":"2023-03-29T14:03:05.246660+0000","last_clean":"2023-03-29T14:03:05.246660+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:05.246660+0000","last_undegraded":"2023-03-29T14:03:05.246660+0000","last_fullsized":"2023-03-29T14:03:05.246660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252204,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":15432,"num_write":50,"num_write_kb":13464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.239315+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:05.239315+0000","last_peered":"2023-03-29T14:03:05.239315+0000","last_clean":"2023-03-29T14:03:05.239315+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:05.239315+0000","last_undegraded":"2023-03-29T14:03:05.239315+0000","last_fullsized":"2023-03-29T14:03:05.239315+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874030,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":20934,"num_write":42,"num_write_kb":12194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.397355+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:04.397355+0000","last_peered":"2023-03-29T14:03:04.397355+0000","last_clean":"2023-03-29T14:03:04.397355+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:04.397355+0000","last_undegraded":"2023-03-29T14:03:04.397355+0000","last_fullsized":"2023-03-29T14:03:04.397355+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'20","reported_seq":84,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.393368+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:05.393368+0000","last_peered":"2023-03-29T14:03:05.393368+0000","last_clean":"2023-03-29T14:03:05.393368+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:05.393368+0000","last_undegraded":"2023-03-29T14:03:05.393368+0000","last_fullsized":"2023-03-29T14:03:05.393368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9815288,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13758,"num_write":40,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.225228+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:05.225228+0000","last_peered":"2023-03-29T14:03:05.225228+0000","last_clean":"2023-03-29T14:03:05.225228+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:05.225228+0000","last_undegraded":"2023-03-29T14:03:05.225228+0000","last_fullsized":"2023-03-29T14:03:05.225228+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":78,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.260393+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:05.260393+0000","last_peered":"2023-03-29T14:03:05.260393+0000","last_clean":"2023-03-29T14:03:05.260393+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:05.260393+0000","last_undegraded":"2023-03-29T14:03:05.260393+0000","last_fullsized":"2023-03-29T14:03:05.260393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":21266,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.387462+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:05.387462+0000","last_peered":"2023-03-29T14:03:05.387462+0000","last_clean":"2023-03-29T14:03:05.387462+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:05.387462+0000","last_undegraded":"2023-03-29T14:03:05.387462+0000","last_fullsized":"2023-03-29T14:03:05.387462+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'34","reported_seq":127,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.416393+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:04.416393+0000","last_peered":"2023-03-29T14:03:04.416393+0000","last_clean":"2023-03-29T14:03:04.416393+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:04.416393+0000","last_undegraded":"2023-03-29T14:03:04.416393+0000","last_fullsized":"2023-03-29T14:03:04.416393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10924484,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":55,"num_read_kb":29857,"num_write":72,"num_write_kb":22142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.248727+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:05.248727+0000","last_peered":"2023-03-29T14:03:05.248727+0000","last_clean":"2023-03-29T14:03:05.248727+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:05.248727+0000","last_undegraded":"2023-03-29T14:03:05.248727+0000","last_fullsized":"2023-03-29T14:03:05.248727+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":272,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.398588+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:04.398588+0000","last_peered":"2023-03-29T14:03:04.398588+0000","last_clean":"2023-03-29T14:03:04.398588+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:04.398588+0000","last_undegraded":"2023-03-29T14:03:04.398588+0000","last_fullsized":"2023-03-29T14:03:04.398588+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"37'40","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.234572+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:05.234572+0000","last_peered":"2023-03-29T14:03:05.234572+0000","last_clean":"2023-03-29T14:03:05.234572+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:05.234572+0000","last_undegraded":"2023-03-29T14:03:05.234572+0000","last_fullsized":"2023-03-29T14:03:05.234572+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21411920,"num_objects":14,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":38262,"num_write":90,"num_write_kb":26440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.230549+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:05.230549+0000","last_peered":"2023-03-29T14:03:05.230549+0000","last_clean":"2023-03-29T14:03:05.230549+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:05.230549+0000","last_undegraded":"2023-03-29T14:03:05.230549+0000","last_fullsized":"2023-03-29T14:03:05.230549+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"55'83","reported_seq":241,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.234994+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:05.234994+0000","last_peered":"2023-03-29T14:03:05.234994+0000","last_clean":"2023-03-29T14:03:05.234994+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:05.234994+0000","last_undegraded":"2023-03-29T14:03:05.234994+0000","last_fullsized":"2023-03-29T14:03:05.234994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28202146,"num_objects":24,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":55627,"num_write":180,"num_write_kb":51182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.405125+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:04.405125+0000","last_peered":"2023-03-29T14:03:04.405125+0000","last_clean":"2023-03-29T14:03:04.405125+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:04.405125+0000","last_undegraded":"2023-03-29T14:03:04.405125+0000","last_fullsized":"2023-03-29T14:03:04.405125+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.396685+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:04.396685+0000","last_peered":"2023-03-29T14:03:04.396685+0000","last_clean":"2023-03-29T14:03:04.396685+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:04.396685+0000","last_undegraded":"2023-03-29T14:03:04.396685+0000","last_fullsized":"2023-03-29T14:03:04.396685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":116,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.409461+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:04.409461+0000","last_peered":"2023-03-29T14:03:04.409461+0000","last_clean":"2023-03-29T14:03:04.409461+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:04.409461+0000","last_undegraded":"2023-03-29T14:03:04.409461+0000","last_fullsized":"2023-03-29T14:03:04.409461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":11149,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:04.396016+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:04.396016+0000","last_peered":"2023-03-29T14:03:04.396016+0000","last_clean":"2023-03-29T14:03:04.396016+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:04.396016+0000","last_undegraded":"2023-03-29T14:03:04.396016+0000","last_fullsized":"2023-03-29T14:03:04.396016+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.247436+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:05.247436+0000","last_peered":"2023-03-29T14:03:05.247436+0000","last_clean":"2023-03-29T14:03:05.247436+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:05.247436+0000","last_undegraded":"2023-03-29T14:03:05.247436+0000","last_fullsized":"2023-03-29T14:03:05.247436+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"48'30","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-03-29T14:03:05.240962+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:05.240962+0000","last_peered":"2023-03-29T14:03:05.240962+0000","last_clean":"2023-03-29T14:03:05.240962+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:05.240962+0000","last_undegraded":"2023-03-29T14:03:05.240962+0000","last_fullsized":"2023-03-29T14:03:05.240962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20235496,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":23295,"num_write":68,"num_write_kb":18062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":232607494,"num_objects":186,"num_object_clones":90,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":12,"num_read":768,"num_read_kb":376752,"num_write":1084,"num_write_kb":310386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":494,"ondisk_log_size":494,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105016,"kb_used_data":98544,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94266824,"statfs":{"total":96636764160,"available":96529227776,"internally_reserved":0,"allocated":100909056,"data_stored":110512261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65456,"kb_used_data":59112,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":94306384,"statfs":{"total":96636764160,"available":96569737216,"internally_reserved":0,"allocated":60530688,"data_stored":64348387,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705724,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":110836,"kb_used_data":103852,"kb_used_omap":0,"kb_used_meta":6976,"kb_avail":94261004,"statfs":{"total":96636764160,"available":96523268096,"internally_reserved":0,"allocated":106344448,"data_stored":116317346,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7143424},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705724,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":101220,"kb_used_data":94492,"kb_used_omap":0,"kb_used_meta":6720,"kb_avail":94270620,"statfs":{"total":96636764160,"available":96533114880,"internally_reserved":0,"allocated":96759808,"data_stored":104124918,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6881280},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:09.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:09.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:09.192+0000 7f0f4dd37700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:09.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:09.196+0000 7f0f4dd37700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:09.385 INFO:tasks.rados.rados.0.smithi017.stdout:280: done (2 left) 2023-03-29T14:03:09.385 INFO:tasks.rados.rados.0.smithi017.stdout:281: write (excl) oid 6 current snap is 24 2023-03-29T14:03:09.385 INFO:tasks.rados.rados.0.smithi017.stdout:278: finishing write tid 1 to smithi017107884-37 2023-03-29T14:03:09.385 INFO:tasks.rados.rados.0.smithi017.stdout:281: seq_num 162 ranges {722734=633751,2063377=659357} 2023-03-29T14:03:09.392 INFO:tasks.rados.rados.0.smithi017.stdout:281: writing smithi017107884-6 from 722734 to 1356485 tid 1 2023-03-29T14:03:09.400 INFO:tasks.rados.rados.0.smithi017.stdout:281: writing smithi017107884-6 from 2063377 to 2722734 tid 2 2023-03-29T14:03:09.400 INFO:tasks.rados.rados.0.smithi017.stdout:282: write oid 7 current snap is 24 2023-03-29T14:03:09.400 INFO:tasks.rados.rados.0.smithi017.stdout:282: seq_num 163 ranges {739541=709000,2121859=426376} 2023-03-29T14:03:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:282: writing smithi017107884-7 from 739541 to 1448541 tid 1 2023-03-29T14:03:09.414 INFO:tasks.rados.rados.0.smithi017.stdout:282: writing smithi017107884-7 from 2121859 to 2548235 tid 2 2023-03-29T14:03:09.414 INFO:tasks.rados.rados.0.smithi017.stdout:283: snap_create 2023-03-29T14:03:09.414 INFO:tasks.rados.rados.0.smithi017.stdout:278: finishing write tid 2 to smithi017107884-37 2023-03-29T14:03:09.414 INFO:tasks.rados.rados.0.smithi017.stdout:279: finishing write tid 1 to smithi017107884-50 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:278: finishing write tid 3 to smithi017107884-37 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:278: finishing write tid 4 to smithi017107884-37 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:278: oid 37 updating version 0 to 24 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:278: oid 37 version 24 is already newer than 21 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:278: oid 37 version 24 is already newer than 24 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 24 (ObjNum 160 snap 24 seq_num 160) dirty exists 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:278: left oid 37 (ObjNum 160 snap 24 seq_num 160) 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:279: finishing write tid 2 to smithi017107884-50 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:279: finishing write tid 3 to smithi017107884-50 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:279: finishing write tid 4 to smithi017107884-50 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:279: oid 50 updating version 0 to 35 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:279: oid 50 updating version 35 to 38 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:279: oid 50 version 38 is already newer than 38 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 38 (ObjNum 161 snap 24 seq_num 161) dirty exists 2023-03-29T14:03:09.415 INFO:tasks.rados.rados.0.smithi017.stdout:279: left oid 50 (ObjNum 161 snap 24 seq_num 161) 2023-03-29T14:03:09.481 INFO:tasks.rados.rados.0.smithi017.stdout:281: finishing write tid 1 to smithi017107884-6 2023-03-29T14:03:09.520 INFO:tasks.rados.rados.0.smithi017.stdout:282: finishing write tid 1 to smithi017107884-7 2023-03-29T14:03:09.520 INFO:tasks.rados.rados.0.smithi017.stdout:281: finishing write tid 2 to smithi017107884-6 2023-03-29T14:03:09.550 INFO:tasks.rados.rados.0.smithi017.stdout:282: finishing write tid 2 to smithi017107884-7 2023-03-29T14:03:09.554 INFO:tasks.rados.rados.0.smithi017.stdout:281: finishing write tid 3 to smithi017107884-6 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:281: finishing write tid 4 to smithi017107884-6 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:281: oid 6 updating version 0 to 41 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:281: oid 6 updating version 41 to 44 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:281: oid 6 version 44 is already newer than 44 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 44 (ObjNum 162 snap 24 seq_num 162) dirty exists 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:281: left oid 6 (ObjNum 162 snap 24 seq_num 162) 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:282: finishing write tid 3 to smithi017107884-7 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:282: finishing write tid 4 to smithi017107884-7 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:282: oid 7 updating version 0 to 22 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:282: oid 7 version 22 is already newer than 22 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:282: oid 7 version 22 is already newer than 22 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 22 (ObjNum 163 snap 24 seq_num 163) dirty exists 2023-03-29T14:03:09.556 INFO:tasks.rados.rados.0.smithi017.stdout:282: left oid 7 (ObjNum 163 snap 24 seq_num 163) 2023-03-29T14:03:10.389 INFO:tasks.rados.rados.0.smithi017.stdout:278: done (4 left) 2023-03-29T14:03:10.390 INFO:tasks.rados.rados.0.smithi017.stdout:279: done (3 left) 2023-03-29T14:03:10.390 INFO:tasks.rados.rados.0.smithi017.stdout:281: done (2 left) 2023-03-29T14:03:10.390 INFO:tasks.rados.rados.0.smithi017.stdout:282: done (1 left) 2023-03-29T14:03:10.390 INFO:tasks.rados.rados.0.smithi017.stdout:283: done (0 left) 2023-03-29T14:03:10.390 INFO:tasks.rados.rados.0.smithi017.stdout:284: read oid 13 snap -1 2023-03-29T14:03:10.390 INFO:tasks.rados.rados.0.smithi017.stdout:284: expect (ObjNum 93 snap 7 seq_num 93) 2023-03-29T14:03:10.390 INFO:tasks.rados.rados.0.smithi017.stdout:285: snap_remove snap 24 2023-03-29T14:03:11.267 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:11.390 INFO:tasks.rados.rados.0.smithi017.stdout:285: done (1 left) 2023-03-29T14:03:11.390 INFO:tasks.rados.rados.0.smithi017.stdout:286: write oid 9 current snap is 25 2023-03-29T14:03:11.390 INFO:tasks.rados.rados.0.smithi017.stdout:286: seq_num 164 ranges {756348=784249,2063988=686387} 2023-03-29T14:03:11.399 INFO:tasks.rados.rados.0.smithi017.stdout:286: writing smithi017107884-9 from 756348 to 1540597 tid 1 2023-03-29T14:03:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:11.402+0000 7f25bf153700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:11.406 INFO:tasks.rados.rados.0.smithi017.stdout:286: writing smithi017107884-9 from 2063988 to 2750375 tid 2 2023-03-29T14:03:11.406 INFO:tasks.rados.rados.0.smithi017.stdout:287: delete oid 26 current snap is 25 2023-03-29T14:03:11.406 INFO:tasks.rados.rados.0.smithi017.stdout:284: expect (ObjNum 93 snap 7 seq_num 93) 2023-03-29T14:03:11.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:11.407+0000 7f25bf153700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:11.628 INFO:tasks.rados.rados.0.smithi017.stdout:286: finishing write tid 1 to smithi017107884-9 2023-03-29T14:03:11.628 INFO:tasks.rados.rados.0.smithi017.stdout:286: finishing write tid 2 to smithi017107884-9 2023-03-29T14:03:11.628 INFO:tasks.rados.rados.0.smithi017.stdout:286: finishing write tid 3 to smithi017107884-9 2023-03-29T14:03:11.628 INFO:tasks.rados.rados.0.smithi017.stdout:286: finishing write tid 4 to smithi017107884-9 2023-03-29T14:03:11.628 INFO:tasks.rados.rados.0.smithi017.stdout:286: oid 9 updating version 0 to 54 2023-03-29T14:03:11.628 INFO:tasks.rados.rados.0.smithi017.stdout:286: oid 9 version 54 is already newer than 54 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:286: oid 9 version 54 is already newer than 54 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 54 (ObjNum 164 snap 25 seq_num 164) dirty exists 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:286: left oid 9 (ObjNum 164 snap 25 seq_num 164) 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:284: done (2 left) 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:286: done (1 left) 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:287: done (0 left) 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:288: read oid 31 snap 23 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:288: expect (ObjNum 30 snap 0 seq_num 30) 2023-03-29T14:03:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:289: snap_create 2023-03-29T14:03:11.641 INFO:tasks.rados.rados.0.smithi017.stdout:288: expect (ObjNum 30 snap 0 seq_num 30) 2023-03-29T14:03:12.397 INFO:tasks.rados.rados.0.smithi017.stdout:288: done (1 left) 2023-03-29T14:03:12.397 INFO:tasks.rados.rados.0.smithi017.stdout:289: done (0 left) 2023-03-29T14:03:12.397 INFO:tasks.rados.rados.0.smithi017.stdout:290: write (excl) oid 7 current snap is 26 2023-03-29T14:03:12.398 INFO:tasks.rados.rados.0.smithi017.stdout:290: seq_num 166 ranges {789962=534747,1780952=690056} 2023-03-29T14:03:12.404 INFO:tasks.rados.rados.0.smithi017.stdout:290: writing smithi017107884-7 from 789962 to 1324709 tid 1 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:290: writing smithi017107884-7 from 1780952 to 2471008 tid 2 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:291: read oid 50 snap -1 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:291: expect (ObjNum 161 snap 24 seq_num 161) 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:292: read oid 36 snap -1 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:292: expect (ObjNum 127 snap 15 seq_num 127) 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:293: read oid 20 snap 23 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:293: expect (ObjNum 153 snap 21 seq_num 153) 2023-03-29T14:03:12.411 INFO:tasks.rados.rados.0.smithi017.stdout:294: delete oid 23 current snap is 26 2023-03-29T14:03:12.548 INFO:tasks.rados.rados.0.smithi017.stdout:291: expect (ObjNum 161 snap 24 seq_num 161) 2023-03-29T14:03:12.723 INFO:tasks.rados.rados.0.smithi017.stdout:293: expect (ObjNum 153 snap 21 seq_num 153) 2023-03-29T14:03:12.895 INFO:tasks.rados.rados.0.smithi017.stdout:292: expect (ObjNum 127 snap 15 seq_num 127) 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: finishing write tid 1 to smithi017107884-7 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: finishing write tid 2 to smithi017107884-7 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: finishing write tid 3 to smithi017107884-7 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: finishing write tid 4 to smithi017107884-7 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: oid 7 updating version 0 to 26 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: oid 7 version 26 is already newer than 23 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: oid 7 version 26 is already newer than 26 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 26 (ObjNum 166 snap 26 seq_num 166) dirty exists 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: left oid 7 (ObjNum 166 snap 26 seq_num 166) 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:290: done (4 left) 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:291: done (3 left) 2023-03-29T14:03:12.977 INFO:tasks.rados.rados.0.smithi017.stdout:292: done (2 left) 2023-03-29T14:03:12.978 INFO:tasks.rados.rados.0.smithi017.stdout:293: done (1 left) 2023-03-29T14:03:12.978 INFO:tasks.rados.rados.0.smithi017.stdout:294: done (0 left) 2023-03-29T14:03:12.978 INFO:tasks.rados.rados.0.smithi017.stdout:295: write oid 29 current snap is 26 2023-03-29T14:03:12.978 INFO:tasks.rados.rados.0.smithi017.stdout:295: seq_num 168 ranges {423576=401598,1497916=693725} 2023-03-29T14:03:12.981 INFO:tasks.rados.rados.0.smithi017.stdout:295: writing smithi017107884-29 from 423576 to 825174 tid 1 2023-03-29T14:03:12.989 INFO:tasks.rados.rados.0.smithi017.stdout:295: writing smithi017107884-29 from 1497916 to 2191641 tid 2 2023-03-29T14:03:12.989 INFO:tasks.rados.rados.0.smithi017.stdout:296: write oid 28 current snap is 26 2023-03-29T14:03:12.989 INFO:tasks.rados.rados.0.smithi017.stdout:296: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2023-03-29T14:03:12.994 INFO:tasks.rados.rados.0.smithi017.stdout:296: writing smithi017107884-28 from 440383 to 917230 tid 1 2023-03-29T14:03:13.000 INFO:tasks.rados.rados.0.smithi017.stdout:296: writing smithi017107884-28 from 1556398 to 1993781 tid 2 2023-03-29T14:03:13.005 INFO:tasks.rados.rados.0.smithi017.stdout:296: writing smithi017107884-28 from 2474721 to 2840383 tid 3 2023-03-29T14:03:13.005 INFO:tasks.rados.rados.0.smithi017.stdout:297: snap_create 2023-03-29T14:03:13.025 INFO:tasks.rados.rados.0.smithi017.stdout:295: finishing write tid 1 to smithi017107884-29 2023-03-29T14:03:13.074 INFO:tasks.rados.rados.0.smithi017.stdout:295: finishing write tid 2 to smithi017107884-29 2023-03-29T14:03:13.076 INFO:tasks.rados.rados.0.smithi017.stdout:295: finishing write tid 3 to smithi017107884-29 2023-03-29T14:03:13.078 INFO:tasks.rados.rados.0.smithi017.stdout:295: finishing write tid 4 to smithi017107884-29 2023-03-29T14:03:13.078 INFO:tasks.rados.rados.0.smithi017.stdout:295: oid 29 updating version 0 to 14 2023-03-29T14:03:13.078 INFO:tasks.rados.rados.0.smithi017.stdout:295: oid 29 version 14 is already newer than 14 2023-03-29T14:03:13.078 INFO:tasks.rados.rados.0.smithi017.stdout:295: oid 29 version 14 is already newer than 14 2023-03-29T14:03:13.078 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 14 (ObjNum 168 snap 26 seq_num 168) dirty exists 2023-03-29T14:03:13.078 INFO:tasks.rados.rados.0.smithi017.stdout:295: left oid 29 (ObjNum 168 snap 26 seq_num 168) 2023-03-29T14:03:13.084 INFO:tasks.rados.rados.0.smithi017.stdout:296: finishing write tid 1 to smithi017107884-28 2023-03-29T14:03:13.101 INFO:tasks.rados.rados.0.smithi017.stdout:296: finishing write tid 2 to smithi017107884-28 2023-03-29T14:03:13.102 INFO:tasks.rados.rados.0.smithi017.stdout:296: finishing write tid 3 to smithi017107884-28 2023-03-29T14:03:13.102 INFO:tasks.rados.rados.0.smithi017.stdout:296: finishing write tid 4 to smithi017107884-28 2023-03-29T14:03:13.103 INFO:tasks.rados.rados.0.smithi017.stdout:296: finishing write tid 5 to smithi017107884-28 2023-03-29T14:03:13.103 INFO:tasks.rados.rados.0.smithi017.stdout:296: oid 28 updating version 0 to 31 2023-03-29T14:03:13.103 INFO:tasks.rados.rados.0.smithi017.stdout:296: oid 28 version 31 is already newer than 31 2023-03-29T14:03:13.104 INFO:tasks.rados.rados.0.smithi017.stdout:296: oid 28 version 31 is already newer than 31 2023-03-29T14:03:13.104 INFO:tasks.rados.rados.0.smithi017.stdout:296: oid 28 version 31 is already newer than 31 2023-03-29T14:03:13.104 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 31 (ObjNum 169 snap 26 seq_num 169) dirty exists 2023-03-29T14:03:13.104 INFO:tasks.rados.rados.0.smithi017.stdout:296: left oid 28 (ObjNum 169 snap 26 seq_num 169) 2023-03-29T14:03:13.179 INFO:tasks.rados.rados.0.smithi017.stdout:295: done (2 left) 2023-03-29T14:03:13.179 INFO:tasks.rados.rados.0.smithi017.stdout:296: done (1 left) 2023-03-29T14:03:13.179 INFO:tasks.rados.rados.0.smithi017.stdout:297: done (0 left) 2023-03-29T14:03:13.179 INFO:tasks.rados.rados.0.smithi017.stdout:298: read oid 24 snap 25 2023-03-29T14:03:13.179 INFO:tasks.rados.rados.0.smithi017.stdout:298: expect (ObjNum 23 snap 0 seq_num 23) 2023-03-29T14:03:13.179 INFO:tasks.rados.rados.0.smithi017.stdout:299: snap_remove snap 23 2023-03-29T14:03:13.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:13.595 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:13.595 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:13.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:13.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:13.659+0000 7fec5b467700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:13.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:13.664+0000 7fec5b467700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:13.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:13.730+0000 7fd8923cb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:13.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:13.737+0000 7fd8923cb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:13.941 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:13.942 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:13.954 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-03-29T14:03:13.177378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":215401702,"num_objects":206,"num_object_clones":106,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":946,"num_read_kb":395746,"num_write":1398,"num_write_kb":325332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":617,"ondisk_log_size":617,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":399956,"kb_used_data":372916,"kb_used_omap":0,"kb_used_meta":27008,"kb_avail":377087404,"statfs":{"total":386547056640,"available":386137501696,"internally_reserved":0,"allocated":381865984,"data_stored":423277046,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":27656192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18386528,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":36,"num_read_kb":18874,"num_write":40,"num_write_kb":11158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.793108"},"pg_stats":[{"pgid":"3.f","version":"49'16","reported_seq":94,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-03-29T14:03:10.408884+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:10.408884+0000","last_peered":"2023-03-29T14:03:10.408884+0000","last_clean":"2023-03-29T14:03:10.408884+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:10.408884+0000","last_undegraded":"2023-03-29T14:03:10.408884+0000","last_fullsized":"2023-03-29T14:03:10.408884+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386594,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":50,"num_read_kb":39297,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":62,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.413557+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:09.413557+0000","last_peered":"2023-03-29T14:03:09.413557+0000","last_clean":"2023-03-29T14:03:09.413557+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:09.413557+0000","last_undegraded":"2023-03-29T14:03:09.413557+0000","last_fullsized":"2023-03-29T14:03:09.413557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"61'55","reported_seq":163,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.421847+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:09.421847+0000","last_peered":"2023-03-29T14:03:09.421847+0000","last_clean":"2023-03-29T14:03:09.421847+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:09.421847+0000","last_undegraded":"2023-03-29T14:03:09.421847+0000","last_fullsized":"2023-03-29T14:03:09.421847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17680898,"num_objects":18,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":42,"num_read_kb":7055,"num_write":114,"num_write_kb":33414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"60'49","reported_seq":164,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.412431+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:09.412431+0000","last_peered":"2023-03-29T14:03:09.412431+0000","last_clean":"2023-03-29T14:03:09.412431+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:09.412431+0000","last_undegraded":"2023-03-29T14:03:09.412431+0000","last_fullsized":"2023-03-29T14:03:09.412431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13848968,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":76,"num_read_kb":31797,"num_write":104,"num_write_kb":29306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":161,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.394443+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:09.394443+0000","last_peered":"2023-03-29T14:03:09.394443+0000","last_clean":"2023-03-29T14:03:09.394443+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:09.394443+0000","last_undegraded":"2023-03-29T14:03:09.394443+0000","last_fullsized":"2023-03-29T14:03:09.394443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":73,"num_read_kb":34884,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.435198+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:09.435198+0000","last_peered":"2023-03-29T14:03:09.435198+0000","last_clean":"2023-03-29T14:03:09.435198+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:09.435198+0000","last_undegraded":"2023-03-29T14:03:09.435198+0000","last_fullsized":"2023-03-29T14:03:09.435198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'22","reported_seq":104,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.429699+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:09.429699+0000","last_peered":"2023-03-29T14:03:09.429699+0000","last_clean":"2023-03-29T14:03:09.429699+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:09.429699+0000","last_undegraded":"2023-03-29T14:03:09.429699+0000","last_fullsized":"2023-03-29T14:03:09.429699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252204,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":15432,"num_write":50,"num_write_kb":13464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"64'22","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.556638+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:09.556638+0000","last_peered":"2023-03-29T14:03:09.556638+0000","last_clean":"2023-03-29T14:03:09.556638+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:09.556638+0000","last_undegraded":"2023-03-29T14:03:09.556638+0000","last_fullsized":"2023-03-29T14:03:09.556638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12100974,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":20935,"num_write":50,"num_write_kb":14414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.399678+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:09.399678+0000","last_peered":"2023-03-29T14:03:09.399678+0000","last_clean":"2023-03-29T14:03:09.399678+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:09.399678+0000","last_undegraded":"2023-03-29T14:03:09.399678+0000","last_fullsized":"2023-03-29T14:03:09.399678+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"63'24","reported_seq":94,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-03-29T14:03:10.419868+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:10.419868+0000","last_peered":"2023-03-29T14:03:10.419868+0000","last_clean":"2023-03-29T14:03:10.419868+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:10.419868+0000","last_undegraded":"2023-03-29T14:03:10.419868+0000","last_fullsized":"2023-03-29T14:03:10.419868+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10386726,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.437378+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:09.437378+0000","last_peered":"2023-03-29T14:03:09.437378+0000","last_clean":"2023-03-29T14:03:09.437378+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:09.437378+0000","last_undegraded":"2023-03-29T14:03:09.437378+0000","last_fullsized":"2023-03-29T14:03:09.437378+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'10","reported_seq":82,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:08.418693+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:08.418693+0000","last_peered":"2023-03-29T14:03:08.418693+0000","last_clean":"2023-03-29T14:03:08.418693+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:08.418693+0000","last_undegraded":"2023-03-29T14:03:08.418693+0000","last_fullsized":"2023-03-29T14:03:08.418693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798382,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":21266,"num_write":26,"num_write_kb":6646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-03-29T14:03:10.418089+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:10.418089+0000","last_peered":"2023-03-29T14:03:10.418089+0000","last_clean":"2023-03-29T14:03:10.418089+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:10.418089+0000","last_undegraded":"2023-03-29T14:03:10.418089+0000","last_fullsized":"2023-03-29T14:03:10.418089+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'38","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.412113+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:09.412113+0000","last_peered":"2023-03-29T14:03:09.412113+0000","last_clean":"2023-03-29T14:03:09.412113+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:09.412113+0000","last_undegraded":"2023-03-29T14:03:09.412113+0000","last_fullsized":"2023-03-29T14:03:09.412113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9680832,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":58,"num_read_kb":29858,"num_write":80,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.432085+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:09.432085+0000","last_peered":"2023-03-29T14:03:09.432085+0000","last_clean":"2023-03-29T14:03:09.432085+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:09.432085+0000","last_undegraded":"2023-03-29T14:03:09.432085+0000","last_fullsized":"2023-03-29T14:03:09.432085+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":276,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.403221+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:09.403221+0000","last_peered":"2023-03-29T14:03:09.403221+0000","last_clean":"2023-03-29T14:03:09.403221+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:09.403221+0000","last_undegraded":"2023-03-29T14:03:09.403221+0000","last_fullsized":"2023-03-29T14:03:09.403221+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"64'44","reported_seq":164,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.555264+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:09.555264+0000","last_peered":"2023-03-29T14:03:09.555264+0000","last_clean":"2023-03-29T14:03:09.555264+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:09.555264+0000","last_undegraded":"2023-03-29T14:03:09.555264+0000","last_fullsized":"2023-03-29T14:03:09.555264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20638864,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":44745,"num_write":98,"num_write_kb":28966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.434624+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:09.434624+0000","last_peered":"2023-03-29T14:03:09.434624+0000","last_clean":"2023-03-29T14:03:09.434624+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:09.434624+0000","last_undegraded":"2023-03-29T14:03:09.434624+0000","last_fullsized":"2023-03-29T14:03:09.434624+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"63'89","reported_seq":254,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:08.429239+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:08.429239+0000","last_peered":"2023-03-29T14:03:08.429239+0000","last_clean":"2023-03-29T14:03:08.429239+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:08.429239+0000","last_undegraded":"2023-03-29T14:03:08.429239+0000","last_fullsized":"2023-03-29T14:03:08.429239+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23933168,"num_objects":28,"num_object_clones":16,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":117,"num_read_kb":55628,"num_write":188,"num_write_kb":53404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.396817+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:09.396817+0000","last_peered":"2023-03-29T14:03:09.396817+0000","last_clean":"2023-03-29T14:03:09.396817+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:09.396817+0000","last_undegraded":"2023-03-29T14:03:09.396817+0000","last_fullsized":"2023-03-29T14:03:09.396817+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":129,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.414680+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:09.414680+0000","last_peered":"2023-03-29T14:03:09.414680+0000","last_clean":"2023-03-29T14:03:09.414680+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:09.414680+0000","last_undegraded":"2023-03-29T14:03:09.414680+0000","last_fullsized":"2023-03-29T14:03:09.414680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:10.438193+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:10.438193+0000","last_peered":"2023-03-29T14:03:10.438193+0000","last_clean":"2023-03-29T14:03:10.438193+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:10.438193+0000","last_undegraded":"2023-03-29T14:03:10.438193+0000","last_fullsized":"2023-03-29T14:03:10.438193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":43,"num_read_kb":18418,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.409653+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:09.409653+0000","last_peered":"2023-03-29T14:03:09.409653+0000","last_clean":"2023-03-29T14:03:09.409653+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:09.409653+0000","last_undegraded":"2023-03-29T14:03:09.409653+0000","last_fullsized":"2023-03-29T14:03:09.409653+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:09.416759+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:09.416759+0000","last_peered":"2023-03-29T14:03:09.416759+0000","last_clean":"2023-03-29T14:03:09.416759+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:09.416759+0000","last_undegraded":"2023-03-29T14:03:09.416759+0000","last_fullsized":"2023-03-29T14:03:09.416759+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"62'32","reported_seq":134,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-03-29T14:03:08.409586+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:08.409586+0000","last_peered":"2023-03-29T14:03:08.409586+0000","last_clean":"2023-03-29T14:03:08.409586+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:08.409586+0000","last_undegraded":"2023-03-29T14:03:08.409586+0000","last_fullsized":"2023-03-29T14:03:08.409586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16000198,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":28413,"num_write":68,"num_write_kb":18062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":214220966,"num_objects":202,"num_object_clones":106,"num_object_copies":606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":804,"num_read_kb":395626,"num_write":1124,"num_write_kb":321544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":520,"ondisk_log_size":520,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":109812,"kb_used_data":103212,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":94262028,"statfs":{"total":96636764160,"available":96524316672,"internally_reserved":0,"allocated":105689088,"data_stored":118163972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":70100,"kb_used_data":63628,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94301740,"statfs":{"total":96636764160,"available":96564981760,"internally_reserved":0,"allocated":65155072,"data_stored":71648677,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115324,"kb_used_data":108212,"kb_used_omap":0,"kb_used_meta":7104,"kb_avail":94256516,"statfs":{"total":96636764160,"available":96518672384,"internally_reserved":0,"allocated":110809088,"data_stored":124004097,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7274496},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":104720,"kb_used_data":97864,"kb_used_omap":0,"kb_used_meta":6848,"kb_avail":94267120,"statfs":{"total":96636764160,"available":96529530880,"internally_reserved":0,"allocated":100212736,"data_stored":109460300,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7012352},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:14.182 INFO:tasks.rados.rados.0.smithi017.stdout:299: done (1 left) 2023-03-29T14:03:14.183 INFO:tasks.rados.rados.0.smithi017.stdout:300: read oid 6 snap -1 2023-03-29T14:03:14.183 INFO:tasks.rados.rados.0.smithi017.stdout:300: expect (ObjNum 162 snap 24 seq_num 162) 2023-03-29T14:03:14.183 INFO:tasks.rados.rados.0.smithi017.stdout:301: snap_remove snap 26 2023-03-29T14:03:15.184 INFO:tasks.rados.rados.0.smithi017.stdout:301: done (2 left) 2023-03-29T14:03:15.184 INFO:tasks.rados.rados.0.smithi017.stdout:302: write oid 31 current snap is 27 2023-03-29T14:03:15.184 INFO:tasks.rados.rados.0.smithi017.stdout:302: seq_num 170 ranges {457190=552096,1614880=581041,2785791=71399} 2023-03-29T14:03:15.189 INFO:tasks.rados.rados.0.smithi017.stdout:302: writing smithi017107884-31 from 457190 to 1009286 tid 1 2023-03-29T14:03:15.196 INFO:tasks.rados.rados.0.smithi017.stdout:302: writing smithi017107884-31 from 1614880 to 2195921 tid 2 2023-03-29T14:03:15.200 INFO:tasks.rados.rados.0.smithi017.stdout:302: writing smithi017107884-31 from 2785791 to 2857190 tid 3 2023-03-29T14:03:15.200 INFO:tasks.rados.rados.0.smithi017.stdout:303: read oid 39 snap -1 2023-03-29T14:03:15.200 INFO:tasks.rados.rados.0.smithi017.stdout:303: expect deleted 2023-03-29T14:03:15.200 INFO:tasks.rados.rados.0.smithi017.stdout:304: write (excl) oid 17 current snap is 27 2023-03-29T14:03:15.200 INFO:tasks.rados.rados.0.smithi017.stdout:298: expect (ObjNum 23 snap 0 seq_num 23) 2023-03-29T14:03:15.294 INFO:tasks.rados.rados.0.smithi017.stdout:300: expect (ObjNum 162 snap 24 seq_num 162) 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: finishing write tid 1 to smithi017107884-31 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: finishing write tid 2 to smithi017107884-31 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: finishing write tid 3 to smithi017107884-31 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: finishing write tid 4 to smithi017107884-31 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: finishing write tid 5 to smithi017107884-31 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: oid 31 updating version 0 to 25 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: oid 31 updating version 25 to 27 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: oid 31 version 27 is already newer than 27 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: oid 31 version 27 is already newer than 26 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 27 (ObjNum 170 snap 27 seq_num 170) dirty exists 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:302: left oid 31 (ObjNum 170 snap 27 seq_num 170) 2023-03-29T14:03:15.442 INFO:tasks.rados.rados.0.smithi017.stdout:304: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2023-03-29T14:03:15.448 INFO:tasks.rados.rados.0.smithi017.stdout:304: writing smithi017107884-17 from 473997 to 1101342 tid 1 2023-03-29T14:03:15.453 INFO:tasks.rados.rados.0.smithi017.stdout:304: writing smithi017107884-17 from 1673362 to 2114414 tid 2 2023-03-29T14:03:15.460 INFO:tasks.rados.rados.0.smithi017.stdout:304: writing smithi017107884-17 from 2529567 to 2873997 tid 3 2023-03-29T14:03:15.460 INFO:tasks.rados.rados.0.smithi017.stdout:298: done (4 left) 2023-03-29T14:03:15.460 INFO:tasks.rados.rados.0.smithi017.stdout:300: done (3 left) 2023-03-29T14:03:15.460 INFO:tasks.rados.rados.0.smithi017.stdout:302: done (2 left) 2023-03-29T14:03:15.460 INFO:tasks.rados.rados.0.smithi017.stdout:303: done (1 left) 2023-03-29T14:03:15.460 INFO:tasks.rados.rados.0.smithi017.stdout:305: snap_create 2023-03-29T14:03:15.502 INFO:tasks.rados.rados.0.smithi017.stdout:304: finishing write tid 1 to smithi017107884-17 2023-03-29T14:03:15.559 INFO:tasks.rados.rados.0.smithi017.stdout:304: finishing write tid 2 to smithi017107884-17 2023-03-29T14:03:15.560 INFO:tasks.rados.rados.0.smithi017.stdout:304: finishing write tid 3 to smithi017107884-17 2023-03-29T14:03:15.561 INFO:tasks.rados.rados.0.smithi017.stdout:304: finishing write tid 4 to smithi017107884-17 2023-03-29T14:03:15.562 INFO:tasks.rados.rados.0.smithi017.stdout:304: finishing write tid 5 to smithi017107884-17 2023-03-29T14:03:15.562 INFO:tasks.rados.rados.0.smithi017.stdout:304: oid 17 updating version 0 to 56 2023-03-29T14:03:15.562 INFO:tasks.rados.rados.0.smithi017.stdout:304: oid 17 updating version 56 to 59 2023-03-29T14:03:15.562 INFO:tasks.rados.rados.0.smithi017.stdout:304: oid 17 version 59 is already newer than 59 2023-03-29T14:03:15.562 INFO:tasks.rados.rados.0.smithi017.stdout:304: oid 17 version 59 is already newer than 59 2023-03-29T14:03:15.562 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 59 (ObjNum 171 snap 27 seq_num 171) dirty exists 2023-03-29T14:03:15.562 INFO:tasks.rados.rados.0.smithi017.stdout:304: left oid 17 (ObjNum 171 snap 27 seq_num 171) 2023-03-29T14:03:15.734 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:15.864 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:15.863+0000 7fb74def5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:15.867 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:15.867+0000 7fb74def5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:16.188 INFO:tasks.rados.rados.0.smithi017.stdout:304: done (1 left) 2023-03-29T14:03:16.188 INFO:tasks.rados.rados.0.smithi017.stdout:305: done (0 left) 2023-03-29T14:03:16.188 INFO:tasks.rados.rados.0.smithi017.stdout:306: read oid 16 snap -1 2023-03-29T14:03:16.189 INFO:tasks.rados.rados.0.smithi017.stdout:306: expect deleted 2023-03-29T14:03:16.189 INFO:tasks.rados.rados.0.smithi017.stdout:307: delete oid 37 current snap is 28 2023-03-29T14:03:16.295 INFO:tasks.rados.rados.0.smithi017.stdout:306: done (1 left) 2023-03-29T14:03:16.295 INFO:tasks.rados.rados.0.smithi017.stdout:307: done (0 left) 2023-03-29T14:03:16.296 INFO:tasks.rados.rados.0.smithi017.stdout:308: snap_remove snap 27 2023-03-29T14:03:17.189 INFO:tasks.rados.rados.0.smithi017.stdout:308: done (0 left) 2023-03-29T14:03:17.189 INFO:tasks.rados.rados.0.smithi017.stdout:309: snap_create 2023-03-29T14:03:17.937 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:18.081 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:18.079+0000 7f3b7d7a7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:18.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:18.084+0000 7f3b7d7a7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:18.194 INFO:tasks.rados.rados.0.smithi017.stdout:309: done (0 left) 2023-03-29T14:03:18.194 INFO:tasks.rados.rados.0.smithi017.stdout:310: write oid 32 current snap is 29 2023-03-29T14:03:18.194 INFO:tasks.rados.rados.0.smithi017.stdout:310: seq_num 173 ranges {507611=777843,2073973=444721} 2023-03-29T14:03:18.201 INFO:tasks.rados.rados.0.smithi017.stdout:310: writing smithi017107884-32 from 507611 to 1285454 tid 1 2023-03-29T14:03:18.208 INFO:tasks.rados.rados.0.smithi017.stdout:310: writing smithi017107884-32 from 2073973 to 2518694 tid 2 2023-03-29T14:03:18.208 INFO:tasks.rados.rados.0.smithi017.stdout:311: read oid 36 snap 28 2023-03-29T14:03:18.208 INFO:tasks.rados.rados.0.smithi017.stdout:311: expect (ObjNum 127 snap 15 seq_num 127) 2023-03-29T14:03:18.208 INFO:tasks.rados.rados.0.smithi017.stdout:312: snap_create 2023-03-29T14:03:18.293 INFO:tasks.rados.rados.0.smithi017.stdout:311: expect (ObjNum 127 snap 15 seq_num 127) 2023-03-29T14:03:18.374 INFO:tasks.rados.rados.0.smithi017.stdout:310: finishing write tid 1 to smithi017107884-32 2023-03-29T14:03:18.374 INFO:tasks.rados.rados.0.smithi017.stdout:310: finishing write tid 2 to smithi017107884-32 2023-03-29T14:03:18.374 INFO:tasks.rados.rados.0.smithi017.stdout:310: finishing write tid 3 to smithi017107884-32 2023-03-29T14:03:18.374 INFO:tasks.rados.rados.0.smithi017.stdout:310: finishing write tid 4 to smithi017107884-32 2023-03-29T14:03:18.374 INFO:tasks.rados.rados.0.smithi017.stdout:310: oid 32 updating version 0 to 94 2023-03-29T14:03:18.375 INFO:tasks.rados.rados.0.smithi017.stdout:310: oid 32 version 94 is already newer than 94 2023-03-29T14:03:18.375 INFO:tasks.rados.rados.0.smithi017.stdout:310: oid 32 version 94 is already newer than 94 2023-03-29T14:03:18.375 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 94 (ObjNum 173 snap 29 seq_num 173) dirty exists 2023-03-29T14:03:18.375 INFO:tasks.rados.rados.0.smithi017.stdout:310: left oid 32 (ObjNum 173 snap 29 seq_num 173) 2023-03-29T14:03:18.959 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:18.959 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:18.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:19.099 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:19.096+0000 7f914362a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:19.102 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:19.103+0000 7f914362a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:19.198 INFO:tasks.rados.rados.0.smithi017.stdout:310: done (2 left) 2023-03-29T14:03:19.198 INFO:tasks.rados.rados.0.smithi017.stdout:311: done (1 left) 2023-03-29T14:03:19.198 INFO:tasks.rados.rados.0.smithi017.stdout:312: done (0 left) 2023-03-29T14:03:19.198 INFO:tasks.rados.rados.0.smithi017.stdout:313: read oid 23 snap -1 2023-03-29T14:03:19.198 INFO:tasks.rados.rados.0.smithi017.stdout:313: expect deleted 2023-03-29T14:03:19.199 INFO:tasks.rados.rados.0.smithi017.stdout:314: snap_create 2023-03-29T14:03:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:19.309 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:19.321 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-03-29T14:03:19.197027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206191664,"num_objects":222,"num_object_clones":122,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1000,"num_read_kb":429543,"num_write":1452,"num_write_kb":340364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":650,"ondisk_log_size":650,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":423216,"kb_used_data":395728,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":377064144,"statfs":{"total":386547056640,"available":386113683456,"internally_reserved":0,"allocated":405225472,"data_stored":455418779,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9210038,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":54,"num_read_kb":33797,"num_write":54,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014211"},"pg_stats":[{"pgid":"3.f","version":"49'16","reported_seq":98,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:16.225847+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:16.225847+0000","last_peered":"2023-03-29T14:03:16.225847+0000","last_clean":"2023-03-29T14:03:16.225847+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:16.225847+0000","last_undegraded":"2023-03-29T14:03:16.225847+0000","last_fullsized":"2023-03-29T14:03:16.225847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386594,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":50,"num_read_kb":39297,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":66,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.212343+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:14.212343+0000","last_peered":"2023-03-29T14:03:14.212343+0000","last_clean":"2023-03-29T14:03:14.212343+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:14.212343+0000","last_undegraded":"2023-03-29T14:03:14.212343+0000","last_fullsized":"2023-03-29T14:03:14.212343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"70'59","reported_seq":177,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:15.509747+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:15.509747+0000","last_peered":"2023-03-29T14:03:15.509747+0000","last_clean":"2023-03-29T14:03:15.509747+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:15.509747+0000","last_undegraded":"2023-03-29T14:03:15.509747+0000","last_fullsized":"2023-03-29T14:03:15.509747+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23428892,"num_objects":18,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":45,"num_read_kb":7056,"num_write":124,"num_write_kb":36176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":176,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.207684+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:14.207684+0000","last_peered":"2023-03-29T14:03:14.207684+0000","last_clean":"2023-03-29T14:03:14.207684+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:14.207684+0000","last_undegraded":"2023-03-29T14:03:14.207684+0000","last_fullsized":"2023-03-29T14:03:14.207684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":168,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.213830+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:14.213830+0000","last_peered":"2023-03-29T14:03:14.213830+0000","last_clean":"2023-03-29T14:03:14.213830+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:14.213830+0000","last_undegraded":"2023-03-29T14:03:14.213830+0000","last_fullsized":"2023-03-29T14:03:14.213830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":40166,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.194054+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:14.194054+0000","last_peered":"2023-03-29T14:03:14.194054+0000","last_clean":"2023-03-29T14:03:14.194054+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:14.194054+0000","last_undegraded":"2023-03-29T14:03:14.194054+0000","last_fullsized":"2023-03-29T14:03:14.194054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"70'27","reported_seq":121,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:15.270111+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:15.270111+0000","last_peered":"2023-03-29T14:03:15.270111+0000","last_clean":"2023-03-29T14:03:15.270111+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:15.270111+0000","last_undegraded":"2023-03-29T14:03:15.270111+0000","last_fullsized":"2023-03-29T14:03:15.270111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958164,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":21526,"num_write":60,"num_write_kb":15820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"67'31","reported_seq":126,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.203839+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:14.203839+0000","last_peered":"2023-03-29T14:03:14.203839+0000","last_clean":"2023-03-29T14:03:14.203839+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:14.203839+0000","last_undegraded":"2023-03-29T14:03:14.203839+0000","last_fullsized":"2023-03-29T14:03:14.203839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":20937,"num_write":68,"num_write_kb":19312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.212944+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:14.212944+0000","last_peered":"2023-03-29T14:03:14.212944+0000","last_clean":"2023-03-29T14:03:14.212944+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:14.212944+0000","last_undegraded":"2023-03-29T14:03:14.212944+0000","last_fullsized":"2023-03-29T14:03:14.212944+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'26","reported_seq":100,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:16.239172+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:16.239172+0000","last_peered":"2023-03-29T14:03:16.239172+0000","last_clean":"2023-03-29T14:03:16.239172+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:16.239172+0000","last_undegraded":"2023-03-29T14:03:16.239172+0000","last_fullsized":"2023-03-29T14:03:16.239172+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5008486,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.208965+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:14.208965+0000","last_peered":"2023-03-29T14:03:14.208965+0000","last_clean":"2023-03-29T14:03:14.208965+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:14.208965+0000","last_undegraded":"2023-03-29T14:03:14.208965+0000","last_fullsized":"2023-03-29T14:03:14.208965+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"67'14","reported_seq":96,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.367010+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:14.367010+0000","last_peered":"2023-03-29T14:03:14.367010+0000","last_clean":"2023-03-29T14:03:14.367010+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:14.367010+0000","last_undegraded":"2023-03-29T14:03:14.367010+0000","last_fullsized":"2023-03-29T14:03:14.367010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420274,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":25450,"num_write":34,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.326406+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:14.326406+0000","last_peered":"2023-03-29T14:03:14.326406+0000","last_clean":"2023-03-29T14:03:14.326406+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:14.326406+0000","last_undegraded":"2023-03-29T14:03:14.326406+0000","last_fullsized":"2023-03-29T14:03:14.326406+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'38","reported_seq":148,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.209942+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:14.209942+0000","last_peered":"2023-03-29T14:03:14.209942+0000","last_clean":"2023-03-29T14:03:14.209942+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:14.209942+0000","last_undegraded":"2023-03-29T14:03:14.209942+0000","last_fullsized":"2023-03-29T14:03:14.209942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9680832,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":72,"num_read_kb":41178,"num_write":80,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.204430+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:14.204430+0000","last_peered":"2023-03-29T14:03:14.204430+0000","last_clean":"2023-03-29T14:03:14.204430+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:14.204430+0000","last_undegraded":"2023-03-29T14:03:14.204430+0000","last_fullsized":"2023-03-29T14:03:14.204430+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":280,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.208096+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:14.208096+0000","last_peered":"2023-03-29T14:03:14.208096+0000","last_clean":"2023-03-29T14:03:14.208096+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:14.208096+0000","last_undegraded":"2023-03-29T14:03:14.208096+0000","last_fullsized":"2023-03-29T14:03:14.208096+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"66'46","reported_seq":174,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.222199+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:14.222199+0000","last_peered":"2023-03-29T14:03:14.222199+0000","last_clean":"2023-03-29T14:03:14.222199+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:14.222199+0000","last_undegraded":"2023-03-29T14:03:14.222199+0000","last_fullsized":"2023-03-29T14:03:14.222199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16605184,"num_objects":18,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":92,"num_read_kb":51658,"num_write":98,"num_write_kb":28966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.195209+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:14.195209+0000","last_peered":"2023-03-29T14:03:14.195209+0000","last_clean":"2023-03-29T14:03:14.195209+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:14.195209+0000","last_undegraded":"2023-03-29T14:03:14.195209+0000","last_fullsized":"2023-03-29T14:03:14.195209+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"67'91","reported_seq":260,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.368799+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:14.368799+0000","last_peered":"2023-03-29T14:03:14.368799+0000","last_clean":"2023-03-29T14:03:14.368799+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:14.368799+0000","last_undegraded":"2023-03-29T14:03:14.368799+0000","last_fullsized":"2023-03-29T14:03:14.368799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19563348,"num_objects":30,"num_object_clones":18,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":119,"num_read_kb":55628,"num_write":188,"num_write_kb":53404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.207366+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:14.207366+0000","last_peered":"2023-03-29T14:03:14.207366+0000","last_clean":"2023-03-29T14:03:14.207366+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:14.207366+0000","last_undegraded":"2023-03-29T14:03:14.207366+0000","last_fullsized":"2023-03-29T14:03:14.207366+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":133,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.201452+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:14.201452+0000","last_peered":"2023-03-29T14:03:14.201452+0000","last_clean":"2023-03-29T14:03:14.201452+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:14.201452+0000","last_undegraded":"2023-03-29T14:03:14.201452+0000","last_fullsized":"2023-03-29T14:03:14.201452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":127,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.197024+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:14.197024+0000","last_peered":"2023-03-29T14:03:14.197024+0000","last_clean":"2023-03-29T14:03:14.197024+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:14.197024+0000","last_undegraded":"2023-03-29T14:03:14.197024+0000","last_fullsized":"2023-03-29T14:03:14.197024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":43,"num_read_kb":18418,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.205521+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:14.205521+0000","last_peered":"2023-03-29T14:03:14.205521+0000","last_clean":"2023-03-29T14:03:14.205521+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:14.205521+0000","last_undegraded":"2023-03-29T14:03:14.205521+0000","last_fullsized":"2023-03-29T14:03:14.205521+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.205826+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:14.205826+0000","last_peered":"2023-03-29T14:03:14.205826+0000","last_clean":"2023-03-29T14:03:14.205826+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:14.205826+0000","last_undegraded":"2023-03-29T14:03:14.205826+0000","last_fullsized":"2023-03-29T14:03:14.205826+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"62'32","reported_seq":138,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-03-29T14:03:14.363643+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:14.363643+0000","last_peered":"2023-03-29T14:03:14.363643+0000","last_clean":"2023-03-29T14:03:14.363643+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:14.363643+0000","last_undegraded":"2023-03-29T14:03:14.363643+0000","last_fullsized":"2023-03-29T14:03:14.363643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16000198,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":28413,"num_write":68,"num_write_kb":18062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":205010928,"num_objects":218,"num_object_clones":122,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":858,"num_read_kb":429423,"num_write":1178,"num_write_kb":336576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":553,"ondisk_log_size":553,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116456,"kb_used_data":109728,"kb_used_omap":0,"kb_used_meta":6720,"kb_avail":94255384,"statfs":{"total":96636764160,"available":96517513216,"internally_reserved":0,"allocated":112361472,"data_stored":127168234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6881280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72452,"kb_used_data":65916,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":94299388,"statfs":{"total":96636764160,"available":96562573312,"internally_reserved":0,"allocated":67497984,"data_stored":74716219,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705734,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121864,"kb_used_data":114624,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94249976,"statfs":{"total":96636764160,"available":96511975424,"internally_reserved":0,"allocated":117374976,"data_stored":133366597,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705734,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":112444,"kb_used_data":105460,"kb_used_omap":0,"kb_used_meta":6976,"kb_avail":94259396,"statfs":{"total":96636764160,"available":96521621504,"internally_reserved":0,"allocated":107991040,"data_stored":120167729,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7143424},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:20.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:20.202 INFO:tasks.rados.rados.0.smithi017.stdout:313: done (1 left) 2023-03-29T14:03:20.202 INFO:tasks.rados.rados.0.smithi017.stdout:314: done (0 left) 2023-03-29T14:03:20.202 INFO:tasks.rados.rados.0.smithi017.stdout:315: snap_remove snap 30 2023-03-29T14:03:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:20.298+0000 7fe4e429c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:20.303+0000 7fe4e429c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:315: done (0 left) 2023-03-29T14:03:21.204 INFO:tasks.rados.rados.0.smithi017.stdout:316: snap_create 2023-03-29T14:03:22.211 INFO:tasks.rados.rados.0.smithi017.stdout:316: done (0 left) 2023-03-29T14:03:22.211 INFO:tasks.rados.rados.0.smithi017.stdout:317: read oid 34 snap -1 2023-03-29T14:03:22.211 INFO:tasks.rados.rados.0.smithi017.stdout:317: expect (ObjNum 94 snap 7 seq_num 94) 2023-03-29T14:03:22.211 INFO:tasks.rados.rados.0.smithi017.stdout:318: delete oid 2 current snap is 32 2023-03-29T14:03:22.303 INFO:tasks.rados.rados.0.smithi017.stdout:317: expect (ObjNum 94 snap 7 seq_num 94) 2023-03-29T14:03:22.373 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:22.497 INFO:tasks.rados.rados.0.smithi017.stdout:317: done (1 left) 2023-03-29T14:03:22.497 INFO:tasks.rados.rados.0.smithi017.stdout:318: done (0 left) 2023-03-29T14:03:22.497 INFO:tasks.rados.rados.0.smithi017.stdout:319: snap_create 2023-03-29T14:03:22.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:22.503+0000 7f1ae7dac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:22.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:22.507+0000 7f1ae7dac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:23.181 INFO:tasks.rados.rados.0.smithi017.stdout:319: done (0 left) 2023-03-29T14:03:23.181 INFO:tasks.rados.rados.0.smithi017.stdout:320: read oid 2 snap -1 2023-03-29T14:03:23.181 INFO:tasks.rados.rados.0.smithi017.stdout:320: expect deleted 2023-03-29T14:03:23.181 INFO:tasks.rados.rados.0.smithi017.stdout:321: write (excl) oid 16 current snap is 33 2023-03-29T14:03:23.181 INFO:tasks.rados.rados.0.smithi017.stdout:321: seq_num 175 ranges {541225=644694,1907290=448390} 2023-03-29T14:03:23.190 INFO:tasks.rados.rados.0.smithi017.stdout:321: writing smithi017107884-16 from 541225 to 1185919 tid 1 2023-03-29T14:03:23.199 INFO:tasks.rados.rados.0.smithi017.stdout:321: writing smithi017107884-16 from 1907290 to 2355680 tid 2 2023-03-29T14:03:23.199 INFO:tasks.rados.rados.0.smithi017.stdout:322: write oid 13 current snap is 33 2023-03-29T14:03:23.199 INFO:tasks.rados.rados.0.smithi017.stdout:322: seq_num 176 ranges {558032=719943,1849419=592048} 2023-03-29T14:03:23.208 INFO:tasks.rados.rados.0.smithi017.stdout:322: writing smithi017107884-13 from 558032 to 1277975 tid 1 2023-03-29T14:03:23.214 INFO:tasks.rados.rados.0.smithi017.stdout:322: writing smithi017107884-13 from 1849419 to 2441467 tid 2 2023-03-29T14:03:23.214 INFO:tasks.rados.rados.0.smithi017.stdout:323: delete oid 22 current snap is 33 2023-03-29T14:03:23.313 INFO:tasks.rados.rados.0.smithi017.stdout:321: finishing write tid 1 to smithi017107884-16 2023-03-29T14:03:23.320 INFO:tasks.rados.rados.0.smithi017.stdout:322: finishing write tid 1 to smithi017107884-13 2023-03-29T14:03:23.321 INFO:tasks.rados.rados.0.smithi017.stdout:322: finishing write tid 2 to smithi017107884-13 2023-03-29T14:03:23.360 INFO:tasks.rados.rados.0.smithi017.stdout:322: finishing write tid 3 to smithi017107884-13 2023-03-29T14:03:23.361 INFO:tasks.rados.rados.0.smithi017.stdout:322: finishing write tid 4 to smithi017107884-13 2023-03-29T14:03:23.361 INFO:tasks.rados.rados.0.smithi017.stdout:322: oid 13 updating version 0 to 35 2023-03-29T14:03:23.361 INFO:tasks.rados.rados.0.smithi017.stdout:322: oid 13 version 35 is already newer than 35 2023-03-29T14:03:23.361 INFO:tasks.rados.rados.0.smithi017.stdout:322: oid 13 version 35 is already newer than 35 2023-03-29T14:03:23.361 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 35 (ObjNum 176 snap 33 seq_num 176) dirty exists 2023-03-29T14:03:23.361 INFO:tasks.rados.rados.0.smithi017.stdout:322: left oid 13 (ObjNum 176 snap 33 seq_num 176) 2023-03-29T14:03:23.364 INFO:tasks.rados.rados.0.smithi017.stdout:321: finishing write tid 2 to smithi017107884-16 2023-03-29T14:03:23.365 INFO:tasks.rados.rados.0.smithi017.stdout:321: finishing write tid 3 to smithi017107884-16 2023-03-29T14:03:23.367 INFO:tasks.rados.rados.0.smithi017.stdout:321: finishing write tid 4 to smithi017107884-16 2023-03-29T14:03:23.367 INFO:tasks.rados.rados.0.smithi017.stdout:321: oid 16 updating version 0 to 17 2023-03-29T14:03:23.367 INFO:tasks.rados.rados.0.smithi017.stdout:321: oid 16 updating version 17 to 19 2023-03-29T14:03:23.367 INFO:tasks.rados.rados.0.smithi017.stdout:321: oid 16 version 19 is already newer than 19 2023-03-29T14:03:23.367 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 19 (ObjNum 175 snap 33 seq_num 175) dirty exists 2023-03-29T14:03:23.367 INFO:tasks.rados.rados.0.smithi017.stdout:321: left oid 16 (ObjNum 175 snap 33 seq_num 175) 2023-03-29T14:03:23.446 INFO:tasks.rados.rados.0.smithi017.stdout:320: done (3 left) 2023-03-29T14:03:23.446 INFO:tasks.rados.rados.0.smithi017.stdout:321: done (2 left) 2023-03-29T14:03:23.447 INFO:tasks.rados.rados.0.smithi017.stdout:322: done (1 left) 2023-03-29T14:03:23.447 INFO:tasks.rados.rados.0.smithi017.stdout:323: done (0 left) 2023-03-29T14:03:23.447 INFO:tasks.rados.rados.0.smithi017.stdout:324: delete oid 35 current snap is 33 2023-03-29T14:03:23.501 INFO:tasks.rados.rados.0.smithi017.stdout:324: done (0 left) 2023-03-29T14:03:23.501 INFO:tasks.rados.rados.0.smithi017.stdout:325: delete oid 46 current snap is 33 2023-03-29T14:03:23.503 INFO:tasks.rados.rados.0.smithi017.stdout:325: done (0 left) 2023-03-29T14:03:23.503 INFO:tasks.rados.rados.0.smithi017.stdout:326: write oid 26 current snap is 33 2023-03-29T14:03:23.503 INFO:tasks.rados.rados.0.smithi017.stdout:326: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2023-03-29T14:03:23.510 INFO:tasks.rados.rados.0.smithi017.stdout:326: writing smithi017107884-26 from 625260 to 1246199 tid 1 2023-03-29T14:03:23.516 INFO:tasks.rados.rados.0.smithi017.stdout:326: writing smithi017107884-26 from 1683347 to 2282733 tid 2 2023-03-29T14:03:23.520 INFO:tasks.rados.rados.0.smithi017.stdout:326: writing smithi017107884-26 from 2943668 to 3025260 tid 3 2023-03-29T14:03:23.520 INFO:tasks.rados.rados.0.smithi017.stdout:327: snap_remove snap 29 2023-03-29T14:03:24.214 INFO:tasks.rados.rados.0.smithi017.stdout:327: done (1 left) 2023-03-29T14:03:24.214 INFO:tasks.rados.rados.0.smithi017.stdout:328: snap_create 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: finishing write tid 1 to smithi017107884-26 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: finishing write tid 2 to smithi017107884-26 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: finishing write tid 3 to smithi017107884-26 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: finishing write tid 4 to smithi017107884-26 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: finishing write tid 5 to smithi017107884-26 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: oid 26 updating version 0 to 50 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: oid 26 version 50 is already newer than 48 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: oid 26 version 50 is already newer than 50 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: oid 26 version 50 is already newer than 49 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 50 (ObjNum 180 snap 33 seq_num 180) dirty exists 2023-03-29T14:03:24.215 INFO:tasks.rados.rados.0.smithi017.stdout:326: left oid 26 (ObjNum 180 snap 33 seq_num 180) 2023-03-29T14:03:24.325 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:24.325 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:24.326 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:24.457+0000 7f84df9c0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:24.464+0000 7f84df9c0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:24.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:24.672 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:24.674 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:24.686 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2023-03-29T14:03:24.213778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":212006886,"num_objects":222,"num_object_clones":122,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":1008,"num_read_kb":433949,"num_write":1460,"num_write_kb":342754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":653,"ondisk_log_size":653,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":426880,"kb_used_data":399392,"kb_used_omap":0,"kb_used_meta":27456,"kb_avail":377060480,"statfs":{"total":386547056640,"available":386109931520,"internally_reserved":0,"allocated":408977408,"data_stored":459138815,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":28114944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5815222,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":8,"num_read_kb":4406,"num_write":8,"num_write_kb":2390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012969"},"pg_stats":[{"pgid":"3.f","version":"49'16","reported_seq":100,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:20.250049+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:20.250049+0000","last_peered":"2023-03-29T14:03:20.250049+0000","last_clean":"2023-03-29T14:03:20.250049+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:20.250049+0000","last_undegraded":"2023-03-29T14:03:20.250049+0000","last_fullsized":"2023-03-29T14:03:20.250049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386594,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":50,"num_read_kb":39297,"num_write":34,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":71,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.238011+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:19.238011+0000","last_peered":"2023-03-29T14:03:19.238011+0000","last_clean":"2023-03-29T14:03:19.238011+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:19.238011+0000","last_undegraded":"2023-03-29T14:03:19.238011+0000","last_fullsized":"2023-03-29T14:03:19.238011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"70'59","reported_seq":179,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.198682+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:17.198682+0000","last_peered":"2023-03-29T14:03:17.198682+0000","last_clean":"2023-03-29T14:03:17.198682+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:17.198682+0000","last_undegraded":"2023-03-29T14:03:17.198682+0000","last_fullsized":"2023-03-29T14:03:17.198682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23428892,"num_objects":18,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":45,"num_read_kb":7056,"num_write":124,"num_write_kb":36176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":178,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.207568+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:17.207568+0000","last_peered":"2023-03-29T14:03:17.207568+0000","last_clean":"2023-03-29T14:03:17.207568+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:17.207568+0000","last_undegraded":"2023-03-29T14:03:17.207568+0000","last_fullsized":"2023-03-29T14:03:17.207568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":173,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.252184+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:19.252184+0000","last_peered":"2023-03-29T14:03:19.252184+0000","last_clean":"2023-03-29T14:03:19.252184+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:19.252184+0000","last_undegraded":"2023-03-29T14:03:19.252184+0000","last_fullsized":"2023-03-29T14:03:19.252184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":40166,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":56,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.204822+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:17.204822+0000","last_peered":"2023-03-29T14:03:17.204822+0000","last_clean":"2023-03-29T14:03:17.204822+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:17.204822+0000","last_undegraded":"2023-03-29T14:03:17.204822+0000","last_fullsized":"2023-03-29T14:03:17.204822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"70'27","reported_seq":123,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.206818+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:17.206818+0000","last_peered":"2023-03-29T14:03:17.206818+0000","last_clean":"2023-03-29T14:03:17.206818+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:17.206818+0000","last_undegraded":"2023-03-29T14:03:17.206818+0000","last_fullsized":"2023-03-29T14:03:17.206818+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958164,"num_objects":12,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":21526,"num_write":60,"num_write_kb":15820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"67'31","reported_seq":128,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.210510+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:17.210510+0000","last_peered":"2023-03-29T14:03:17.210510+0000","last_clean":"2023-03-29T14:03:17.210510+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:17.210510+0000","last_undegraded":"2023-03-29T14:03:17.210510+0000","last_fullsized":"2023-03-29T14:03:17.210510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":20937,"num_write":68,"num_write_kb":19312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.262552+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:19.262552+0000","last_peered":"2023-03-29T14:03:19.262552+0000","last_clean":"2023-03-29T14:03:19.262552+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:19.262552+0000","last_undegraded":"2023-03-29T14:03:19.262552+0000","last_fullsized":"2023-03-29T14:03:19.262552+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'26","reported_seq":102,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:20.247170+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:20.247170+0000","last_peered":"2023-03-29T14:03:20.247170+0000","last_clean":"2023-03-29T14:03:20.247170+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:20.247170+0000","last_undegraded":"2023-03-29T14:03:20.247170+0000","last_fullsized":"2023-03-29T14:03:20.247170+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5008486,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":59,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.200684+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:17.200684+0000","last_peered":"2023-03-29T14:03:17.200684+0000","last_clean":"2023-03-29T14:03:17.200684+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:17.200684+0000","last_undegraded":"2023-03-29T14:03:17.200684+0000","last_fullsized":"2023-03-29T14:03:17.200684+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"67'14","reported_seq":98,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-03-29T14:03:18.262096+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:18.262096+0000","last_peered":"2023-03-29T14:03:18.262096+0000","last_clean":"2023-03-29T14:03:18.262096+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:18.262096+0000","last_undegraded":"2023-03-29T14:03:18.262096+0000","last_fullsized":"2023-03-29T14:03:18.262096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420274,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":25450,"num_write":34,"num_write_kb":8788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:20.258166+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:20.258166+0000","last_peered":"2023-03-29T14:03:20.258166+0000","last_clean":"2023-03-29T14:03:20.258166+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:20.258166+0000","last_undegraded":"2023-03-29T14:03:20.258166+0000","last_fullsized":"2023-03-29T14:03:20.258166+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'38","reported_seq":156,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.263791+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:19.263791+0000","last_peered":"2023-03-29T14:03:19.263791+0000","last_clean":"2023-03-29T14:03:19.263791+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:19.263791+0000","last_undegraded":"2023-03-29T14:03:19.263791+0000","last_fullsized":"2023-03-29T14:03:19.263791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9680832,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":79,"num_read_kb":45583,"num_write":80,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":59,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.206178+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:17.206178+0000","last_peered":"2023-03-29T14:03:17.206178+0000","last_clean":"2023-03-29T14:03:17.206178+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:17.206178+0000","last_undegraded":"2023-03-29T14:03:17.206178+0000","last_fullsized":"2023-03-29T14:03:17.206178+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":285,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.219223+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:19.219223+0000","last_peered":"2023-03-29T14:03:19.219223+0000","last_clean":"2023-03-29T14:03:19.219223+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:19.219223+0000","last_undegraded":"2023-03-29T14:03:19.219223+0000","last_fullsized":"2023-03-29T14:03:19.219223+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"66'46","reported_seq":176,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.202733+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:17.202733+0000","last_peered":"2023-03-29T14:03:17.202733+0000","last_clean":"2023-03-29T14:03:17.202733+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:17.202733+0000","last_undegraded":"2023-03-29T14:03:17.202733+0000","last_fullsized":"2023-03-29T14:03:17.202733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16605184,"num_objects":18,"num_object_clones":10,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":92,"num_read_kb":51658,"num_write":98,"num_write_kb":28966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.205516+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:17.205516+0000","last_peered":"2023-03-29T14:03:17.205516+0000","last_clean":"2023-03-29T14:03:17.205516+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:17.205516+0000","last_undegraded":"2023-03-29T14:03:17.205516+0000","last_fullsized":"2023-03-29T14:03:17.205516+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"73'94","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-03-29T14:03:18.345231+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:18.345231+0000","last_peered":"2023-03-29T14:03:18.345231+0000","last_clean":"2023-03-29T14:03:18.345231+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:18.345231+0000","last_undegraded":"2023-03-29T14:03:18.345231+0000","last_fullsized":"2023-03-29T14:03:18.345231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25378570,"num_objects":30,"num_object_clones":18,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":55629,"num_write":196,"num_write_kb":55794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.234012+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:19.234012+0000","last_peered":"2023-03-29T14:03:19.234012+0000","last_clean":"2023-03-29T14:03:19.234012+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:19.234012+0000","last_undegraded":"2023-03-29T14:03:19.234012+0000","last_fullsized":"2023-03-29T14:03:19.234012+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"35'32","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.239402+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:19.239402+0000","last_peered":"2023-03-29T14:03:19.239402+0000","last_clean":"2023-03-29T14:03:19.239402+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:19.239402+0000","last_undegraded":"2023-03-29T14:03:19.239402+0000","last_fullsized":"2023-03-29T14:03:19.239402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15227010,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"36'31","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.254825+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:19.254825+0000","last_peered":"2023-03-29T14:03:19.254825+0000","last_clean":"2023-03-29T14:03:19.254825+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:19.254825+0000","last_undegraded":"2023-03-29T14:03:19.254825+0000","last_fullsized":"2023-03-29T14:03:19.254825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20470728,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":43,"num_read_kb":18418,"num_write":70,"num_write_kb":20466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-03-29T14:03:19.253332+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:19.253332+0000","last_peered":"2023-03-29T14:03:19.253332+0000","last_clean":"2023-03-29T14:03:19.253332+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:19.253332+0000","last_undegraded":"2023-03-29T14:03:19.253332+0000","last_fullsized":"2023-03-29T14:03:19.253332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":59,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-03-29T14:03:17.196562+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:17.196562+0000","last_peered":"2023-03-29T14:03:17.196562+0000","last_clean":"2023-03-29T14:03:17.196562+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:17.196562+0000","last_undegraded":"2023-03-29T14:03:17.196562+0000","last_fullsized":"2023-03-29T14:03:17.196562+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"62'32","reported_seq":140,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-03-29T14:03:18.242065+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:18.242065+0000","last_peered":"2023-03-29T14:03:18.242065+0000","last_clean":"2023-03-29T14:03:18.242065+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:18.242065+0000","last_undegraded":"2023-03-29T14:03:18.242065+0000","last_fullsized":"2023-03-29T14:03:18.242065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16000198,"num_objects":12,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":28413,"num_write":68,"num_write_kb":18062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":210826150,"num_objects":218,"num_object_clones":122,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":866,"num_read_kb":433829,"num_write":1186,"num_write_kb":338966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":556,"ondisk_log_size":556,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":117664,"kb_used_data":110936,"kb_used_omap":0,"kb_used_meta":6720,"kb_avail":94254176,"statfs":{"total":96636764160,"available":96516276224,"internally_reserved":0,"allocated":113598464,"data_stored":128397332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6881280},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73672,"kb_used_data":67136,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":94298168,"statfs":{"total":96636764160,"available":96561324032,"internally_reserved":0,"allocated":68747264,"data_stored":75955160,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123076,"kb_used_data":115836,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94248764,"statfs":{"total":96636764160,"available":96510734336,"internally_reserved":0,"allocated":118616064,"data_stored":134598972,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705739,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":112468,"kb_used_data":105484,"kb_used_omap":0,"kb_used_meta":6976,"kb_avail":94259372,"statfs":{"total":96636764160,"available":96521596928,"internally_reserved":0,"allocated":108015616,"data_stored":120187351,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7143424},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:24.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:24.707+0000 7fd5c754a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:24.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:24.711+0000 7fd5c754a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:25.216 INFO:tasks.rados.rados.0.smithi017.stdout:326: done (1 left) 2023-03-29T14:03:25.216 INFO:tasks.rados.rados.0.smithi017.stdout:328: done (0 left) 2023-03-29T14:03:25.217 INFO:tasks.rados.rados.0.smithi017.stdout:329: write (excl) oid 29 current snap is 34 2023-03-29T14:03:25.217 INFO:tasks.rados.rados.0.smithi017.stdout:329: seq_num 181 ranges {642067=696188,2025476=459397,2971091=70976} 2023-03-29T14:03:25.223 INFO:tasks.rados.rados.0.smithi017.stdout:329: writing smithi017107884-29 from 642067 to 1338255 tid 1 2023-03-29T14:03:25.230 INFO:tasks.rados.rados.0.smithi017.stdout:329: writing smithi017107884-29 from 2025476 to 2484873 tid 2 2023-03-29T14:03:25.233 INFO:tasks.rados.rados.0.smithi017.stdout:329: writing smithi017107884-29 from 2971091 to 3042067 tid 3 2023-03-29T14:03:25.233 INFO:tasks.rados.rados.0.smithi017.stdout:330: snap_remove snap 25 2023-03-29T14:03:26.222 INFO:tasks.rados.rados.0.smithi017.stdout:330: done (1 left) 2023-03-29T14:03:26.223 INFO:tasks.rados.rados.0.smithi017.stdout:331: write (excl) oid 28 current snap is 34 2023-03-29T14:03:26.223 INFO:tasks.rados.rados.0.smithi017.stdout:329: finishing write tid 1 to smithi017107884-29 2023-03-29T14:03:26.223 INFO:tasks.rados.rados.0.smithi017.stdout:331: seq_num 182 ranges {658874=771437,2083958=603055} 2023-03-29T14:03:26.230 INFO:tasks.rados.rados.0.smithi017.stdout:331: writing smithi017107884-28 from 658874 to 1430311 tid 1 2023-03-29T14:03:26.237 INFO:tasks.rados.rados.0.smithi017.stdout:331: writing smithi017107884-28 from 2083958 to 2687013 tid 2 2023-03-29T14:03:26.238 INFO:tasks.rados.rados.0.smithi017.stdout:332: write oid 19 current snap is 34 2023-03-29T14:03:26.238 INFO:tasks.rados.rados.0.smithi017.stdout:329: finishing write tid 2 to smithi017107884-29 2023-03-29T14:03:26.238 INFO:tasks.rados.rados.0.smithi017.stdout:329: finishing write tid 3 to smithi017107884-29 2023-03-29T14:03:26.238 INFO:tasks.rados.rados.0.smithi017.stdout:329: finishing write tid 4 to smithi017107884-29 2023-03-29T14:03:26.238 INFO:tasks.rados.rados.0.smithi017.stdout:329: finishing write tid 5 to smithi017107884-29 2023-03-29T14:03:26.238 INFO:tasks.rados.rados.0.smithi017.stdout:329: oid 29 updating version 0 to 15 2023-03-29T14:03:26.238 INFO:tasks.rados.rados.0.smithi017.stdout:329: oid 29 updating version 15 to 19 2023-03-29T14:03:26.239 INFO:tasks.rados.rados.0.smithi017.stdout:329: oid 29 version 19 is already newer than 19 2023-03-29T14:03:26.239 INFO:tasks.rados.rados.0.smithi017.stdout:329: oid 29 version 19 is already newer than 19 2023-03-29T14:03:26.239 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 19 (ObjNum 181 snap 34 seq_num 181) dirty exists 2023-03-29T14:03:26.239 INFO:tasks.rados.rados.0.smithi017.stdout:329: left oid 29 (ObjNum 181 snap 34 seq_num 181) 2023-03-29T14:03:26.239 INFO:tasks.rados.rados.0.smithi017.stdout:332: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2023-03-29T14:03:26.244 INFO:tasks.rados.rados.0.smithi017.stdout:332: writing smithi017107884-19 from 675681 to 1238720 tid 1 2023-03-29T14:03:26.252 INFO:tasks.rados.rados.0.smithi017.stdout:332: writing smithi017107884-19 from 1858793 to 2605506 tid 2 2023-03-29T14:03:26.254 INFO:tasks.rados.rados.0.smithi017.stdout:332: writing smithi017107884-19 from 3025937 to 3075681 tid 3 2023-03-29T14:03:26.255 INFO:tasks.rados.rados.0.smithi017.stdout:329: done (2 left) 2023-03-29T14:03:26.255 INFO:tasks.rados.rados.0.smithi017.stdout:333: read oid 14 snap 32 2023-03-29T14:03:26.255 INFO:tasks.rados.rados.0.smithi017.stdout:333: expect (ObjNum 139 snap 17 seq_num 139) 2023-03-29T14:03:26.255 INFO:tasks.rados.rados.0.smithi017.stdout:334: snap_remove snap 33 2023-03-29T14:03:26.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:26.915 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:26.914+0000 7f7402bb4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:26.918+0000 7f7402bb4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:27.226 INFO:tasks.rados.rados.0.smithi017.stdout:334: done (3 left) 2023-03-29T14:03:27.227 INFO:tasks.rados.rados.0.smithi017.stdout:335: write oid 33 current snap is 34 2023-03-29T14:03:27.227 INFO:tasks.rados.rados.0.smithi017.stdout:335: seq_num 184 ranges {692488=638288,1917275=606724} 2023-03-29T14:03:27.233 INFO:tasks.rados.rados.0.smithi017.stdout:335: writing smithi017107884-33 from 692488 to 1330776 tid 1 2023-03-29T14:03:27.240 INFO:tasks.rados.rados.0.smithi017.stdout:335: writing smithi017107884-33 from 1917275 to 2523999 tid 2 2023-03-29T14:03:27.240 INFO:tasks.rados.rados.0.smithi017.stdout:336: read oid 4 snap -1 2023-03-29T14:03:27.240 INFO:tasks.rados.rados.0.smithi017.stdout:336: expect (ObjNum 117 snap 10 seq_num 117) 2023-03-29T14:03:27.241 INFO:tasks.rados.rados.0.smithi017.stdout:337: write (excl) oid 12 current snap is 34 2023-03-29T14:03:27.241 INFO:tasks.rados.rados.0.smithi017.stdout:337: seq_num 185 ranges {709295=713537,1859404=750382} 2023-03-29T14:03:27.248 INFO:tasks.rados.rados.0.smithi017.stdout:337: writing smithi017107884-12 from 709295 to 1422832 tid 1 2023-03-29T14:03:27.255 INFO:tasks.rados.rados.0.smithi017.stdout:337: writing smithi017107884-12 from 1859404 to 2609786 tid 2 2023-03-29T14:03:27.255 INFO:tasks.rados.rados.0.smithi017.stdout:338: snap_remove snap 28 2023-03-29T14:03:28.229 INFO:tasks.rados.rados.0.smithi017.stdout:338: done (6 left) 2023-03-29T14:03:28.230 INFO:tasks.rados.rados.0.smithi017.stdout:339: read oid 1 snap -1 2023-03-29T14:03:28.230 INFO:tasks.rados.rados.0.smithi017.stdout:339: expect (ObjNum 96 snap 7 seq_num 96) 2023-03-29T14:03:28.230 INFO:tasks.rados.rados.0.smithi017.stdout:340: snap_create 2023-03-29T14:03:28.230 INFO:tasks.rados.rados.0.smithi017.stdout:333: expect (ObjNum 139 snap 17 seq_num 139) 2023-03-29T14:03:28.343 INFO:tasks.rados.rados.0.smithi017.stdout:331: finishing write tid 1 to smithi017107884-28 2023-03-29T14:03:28.343 INFO:tasks.rados.rados.0.smithi017.stdout:332: finishing write tid 1 to smithi017107884-19 2023-03-29T14:03:28.343 INFO:tasks.rados.rados.0.smithi017.stdout:332: finishing write tid 2 to smithi017107884-19 2023-03-29T14:03:28.343 INFO:tasks.rados.rados.0.smithi017.stdout:332: finishing write tid 3 to smithi017107884-19 2023-03-29T14:03:28.343 INFO:tasks.rados.rados.0.smithi017.stdout:332: finishing write tid 4 to smithi017107884-19 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:332: finishing write tid 5 to smithi017107884-19 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:332: oid 19 updating version 0 to 37 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:332: oid 19 version 37 is already newer than 37 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:332: oid 19 version 37 is already newer than 37 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:332: oid 19 version 37 is already newer than 37 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 37 (ObjNum 183 snap 34 seq_num 183) dirty exists 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:332: left oid 19 (ObjNum 183 snap 34 seq_num 183) 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:331: finishing write tid 2 to smithi017107884-28 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:331: finishing write tid 3 to smithi017107884-28 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:331: finishing write tid 4 to smithi017107884-28 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:331: oid 28 updating version 0 to 35 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:331: oid 28 version 35 is already newer than 32 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:331: oid 28 version 35 is already newer than 35 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 35 (ObjNum 182 snap 34 seq_num 182) dirty exists 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:331: left oid 28 (ObjNum 182 snap 34 seq_num 182) 2023-03-29T14:03:28.344 INFO:tasks.rados.rados.0.smithi017.stdout:336: expect (ObjNum 117 snap 10 seq_num 117) 2023-03-29T14:03:28.596 INFO:tasks.rados.rados.0.smithi017.stdout:335: finishing write tid 1 to smithi017107884-33 2023-03-29T14:03:28.596 INFO:tasks.rados.rados.0.smithi017.stdout:337: finishing write tid 1 to smithi017107884-12 2023-03-29T14:03:28.596 INFO:tasks.rados.rados.0.smithi017.stdout:335: finishing write tid 2 to smithi017107884-33 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:335: finishing write tid 3 to smithi017107884-33 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:335: finishing write tid 4 to smithi017107884-33 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:335: oid 33 updating version 0 to 41 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:335: oid 33 version 41 is already newer than 41 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:335: oid 33 version 41 is already newer than 41 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 41 (ObjNum 184 snap 34 seq_num 184) dirty exists 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:335: left oid 33 (ObjNum 184 snap 34 seq_num 184) 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:337: finishing write tid 2 to smithi017107884-12 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:337: finishing write tid 3 to smithi017107884-12 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:337: finishing write tid 4 to smithi017107884-12 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:337: oid 12 updating version 0 to 36 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:337: oid 12 updating version 36 to 38 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:337: oid 12 version 38 is already newer than 38 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 38 (ObjNum 185 snap 34 seq_num 185) dirty exists 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:337: left oid 12 (ObjNum 185 snap 34 seq_num 185) 2023-03-29T14:03:28.597 INFO:tasks.rados.rados.0.smithi017.stdout:339: expect (ObjNum 96 snap 7 seq_num 96) 2023-03-29T14:03:28.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:29.119 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:29.118+0000 7faeb7bf2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:29.122 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:29.122+0000 7faeb7bf2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:29.230 INFO:tasks.rados.rados.0.smithi017.stdout:331: done (7 left) 2023-03-29T14:03:29.230 INFO:tasks.rados.rados.0.smithi017.stdout:332: done (6 left) 2023-03-29T14:03:29.231 INFO:tasks.rados.rados.0.smithi017.stdout:333: done (5 left) 2023-03-29T14:03:29.231 INFO:tasks.rados.rados.0.smithi017.stdout:335: done (4 left) 2023-03-29T14:03:29.231 INFO:tasks.rados.rados.0.smithi017.stdout:336: done (3 left) 2023-03-29T14:03:29.231 INFO:tasks.rados.rados.0.smithi017.stdout:337: done (2 left) 2023-03-29T14:03:29.231 INFO:tasks.rados.rados.0.smithi017.stdout:339: done (1 left) 2023-03-29T14:03:29.231 INFO:tasks.rados.rados.0.smithi017.stdout:340: done (0 left) 2023-03-29T14:03:29.231 INFO:tasks.rados.rados.0.smithi017.stdout:341: snap_create 2023-03-29T14:03:29.689 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:29.690 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:03:29.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:29.818+0000 7f837fac5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:29.826+0000 7f837fac5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:30.236 INFO:tasks.rados.rados.0.smithi017.stdout:341: done (0 left) 2023-03-29T14:03:30.236 INFO:tasks.rados.rados.0.smithi017.stdout:342: read oid 24 snap 32 2023-03-29T14:03:30.236 INFO:tasks.rados.rados.0.smithi017.stdout:342: expect (ObjNum 23 snap 0 seq_num 23) 2023-03-29T14:03:30.236 INFO:tasks.rados.rados.0.smithi017.stdout:343: delete oid 27 current snap is 36 2023-03-29T14:03:30.239 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:03:30.251 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:03:30.279 INFO:tasks.rados.rados.0.smithi017.stdout:342: expect (ObjNum 23 snap 0 seq_num 23) 2023-03-29T14:03:30.374 INFO:tasks.rados.rados.0.smithi017.stdout:342: done (1 left) 2023-03-29T14:03:30.374 INFO:tasks.rados.rados.0.smithi017.stdout:343: done (0 left) 2023-03-29T14:03:30.374 INFO:tasks.rados.rados.0.smithi017.stdout:344: write (excl) oid 14 current snap is 36 2023-03-29T14:03:30.374 INFO:tasks.rados.rados.0.smithi017.stdout:344: seq_num 187 ranges {742909=464035,1976368=754051} 2023-03-29T14:03:30.381 INFO:tasks.rados.rados.0.smithi017.stdout:344: writing smithi017107884-14 from 742909 to 1206944 tid 1 2023-03-29T14:03:30.390 INFO:tasks.rados.rados.0.smithi017.stdout:344: writing smithi017107884-14 from 1976368 to 2730419 tid 2 2023-03-29T14:03:30.390 INFO:tasks.rados.rados.0.smithi017.stdout:345: snap_create 2023-03-29T14:03:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:30.385+0000 7f0ae2b0a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:30.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:30.394+0000 7f0ae2b0a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:30.422 INFO:tasks.rados.rados.0.smithi017.stdout:344: finishing write tid 1 to smithi017107884-14 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:344: finishing write tid 2 to smithi017107884-14 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:344: finishing write tid 3 to smithi017107884-14 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:344: finishing write tid 4 to smithi017107884-14 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:344: oid 14 updating version 0 to 31 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:344: oid 14 version 31 is already newer than 31 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:344: oid 14 version 31 is already newer than 28 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 31 (ObjNum 187 snap 36 seq_num 187) dirty exists 2023-03-29T14:03:30.463 INFO:tasks.rados.rados.0.smithi017.stdout:344: left oid 14 (ObjNum 187 snap 36 seq_num 187) 2023-03-29T14:03:31.195 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:31.241 INFO:tasks.rados.rados.0.smithi017.stdout:344: done (1 left) 2023-03-29T14:03:31.241 INFO:tasks.rados.rados.0.smithi017.stdout:345: done (0 left) 2023-03-29T14:03:31.242 INFO:tasks.rados.rados.0.smithi017.stdout:346: write (excl) oid 13 current snap is 37 2023-03-29T14:03:31.242 INFO:tasks.rados.rados.0.smithi017.stdout:346: seq_num 188 ranges {759716=539284,2034850=614062} 2023-03-29T14:03:31.246 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:03:31.248 INFO:tasks.rados.rados.0.smithi017.stdout:346: writing smithi017107884-13 from 759716 to 1299000 tid 1 2023-03-29T14:03:31.256 INFO:tasks.rados.rados.0.smithi017.stdout:346: writing smithi017107884-13 from 2034850 to 2648912 tid 2 2023-03-29T14:03:31.256 INFO:tasks.rados.rados.0.smithi017.stdout:347: snap_create 2023-03-29T14:03:31.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:03:31.348 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:31.347+0000 7ff3b9f81700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:31.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:31.352+0000 7ff3b9f81700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:31.409 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:31.404+0000 7f7439c67700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:31.412+0000 7f7439c67700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:31.559 INFO:tasks.rados.rados.0.smithi017.stdout:346: finishing write tid 1 to smithi017107884-13 2023-03-29T14:03:31.582 INFO:tasks.rados.rados.0.smithi017.stdout:346: finishing write tid 2 to smithi017107884-13 2023-03-29T14:03:31.585 INFO:tasks.rados.rados.0.smithi017.stdout:346: finishing write tid 3 to smithi017107884-13 2023-03-29T14:03:31.587 INFO:tasks.rados.rados.0.smithi017.stdout:346: finishing write tid 4 to smithi017107884-13 2023-03-29T14:03:31.587 INFO:tasks.rados.rados.0.smithi017.stdout:346: oid 13 updating version 0 to 42 2023-03-29T14:03:31.587 INFO:tasks.rados.rados.0.smithi017.stdout:346: oid 13 version 42 is already newer than 42 2023-03-29T14:03:31.587 INFO:tasks.rados.rados.0.smithi017.stdout:346: oid 13 version 42 is already newer than 39 2023-03-29T14:03:31.587 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 42 (ObjNum 188 snap 37 seq_num 188) dirty exists 2023-03-29T14:03:31.587 INFO:tasks.rados.rados.0.smithi017.stdout:346: left oid 13 (ObjNum 188 snap 37 seq_num 188) 2023-03-29T14:03:32.244 INFO:tasks.rados.rados.0.smithi017.stdout:346: done (1 left) 2023-03-29T14:03:32.244 INFO:tasks.rados.rados.0.smithi017.stdout:347: done (0 left) 2023-03-29T14:03:32.245 INFO:tasks.rados.rados.0.smithi017.stdout:348: read oid 49 snap -1 2023-03-29T14:03:32.245 INFO:tasks.rados.rados.0.smithi017.stdout:348: expect (ObjNum 142 snap 18 seq_num 142) 2023-03-29T14:03:32.245 INFO:tasks.rados.rados.0.smithi017.stdout:349: read oid 33 snap 37 2023-03-29T14:03:32.245 INFO:tasks.rados.rados.0.smithi017.stdout:349: expect (ObjNum 184 snap 34 seq_num 184) 2023-03-29T14:03:32.245 INFO:tasks.rados.rados.0.smithi017.stdout:350: read oid 38 snap -1 2023-03-29T14:03:32.245 INFO:tasks.rados.rados.0.smithi017.stdout:350: expect (ObjNum 132 snap 15 seq_num 132) 2023-03-29T14:03:32.245 INFO:tasks.rados.rados.0.smithi017.stdout:351: snap_remove snap 35 2023-03-29T14:03:32.247 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:03:32.261 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:03:32.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:32.399+0000 7f576fe51700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:32.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:32.408+0000 7f576fe51700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:33.186 INFO:tasks.rados.rados.0.smithi017.stdout:351: done (3 left) 2023-03-29T14:03:33.186 INFO:tasks.rados.rados.0.smithi017.stdout:352: snap_remove snap 31 2023-03-29T14:03:33.186 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:03:33.198 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:03:33.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:33.332 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:33.329+0000 7f6166616700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:33.335+0000 7f6166616700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:33.540 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:33.542 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:33.554 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2023-03-29T14:03:33.182027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209183574,"num_objects":244,"num_object_clones":142,"num_object_copies":732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1068,"num_read_kb":469863,"num_write":1538,"num_write_kb":365420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":695,"ondisk_log_size":695,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":460748,"kb_used_data":432620,"kb_used_omap":0,"kb_used_meta":28096,"kb_avail":377026612,"statfs":{"total":386547056640,"available":386075250688,"internally_reserved":0,"allocated":443002880,"data_stored":503743049,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":28770304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5344692,"num_objects":12,"num_object_clones":10,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":41,"num_read_kb":28784,"num_write":42,"num_write_kb":13020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.952688"},"pg_stats":[{"pgid":"3.f","version":"78'21","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472865+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:29.472865+0000","last_peered":"2023-03-29T14:03:29.472865+0000","last_clean":"2023-03-29T14:03:29.472865+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:29.472865+0000","last_undegraded":"2023-03-29T14:03:29.472865+0000","last_fullsized":"2023-03-29T14:03:29.472865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5882450,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":39298,"num_write":42,"num_write_kb":10896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":78,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.320023+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:30.320023+0000","last_peered":"2023-03-29T14:03:30.320023+0000","last_clean":"2023-03-29T14:03:30.320023+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:30.320023+0000","last_undegraded":"2023-03-29T14:03:30.320023+0000","last_fullsized":"2023-03-29T14:03:30.320023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"70'59","reported_seq":191,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.254333+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:29.254333+0000","last_peered":"2023-03-29T14:03:29.254333+0000","last_clean":"2023-03-29T14:03:29.254333+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:29.254333+0000","last_undegraded":"2023-03-29T14:03:29.254333+0000","last_fullsized":"2023-03-29T14:03:29.254333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23428892,"num_objects":18,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":15374,"num_write":124,"num_write_kb":36176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":187,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.255581+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:29.255581+0000","last_peered":"2023-03-29T14:03:29.255581+0000","last_clean":"2023-03-29T14:03:29.255581+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:29.255581+0000","last_undegraded":"2023-03-29T14:03:29.255581+0000","last_fullsized":"2023-03-29T14:03:29.255581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.504932+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:29.504932+0000","last_peered":"2023-03-29T14:03:29.504932+0000","last_clean":"2023-03-29T14:03:29.504932+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:29.504932+0000","last_undegraded":"2023-03-29T14:03:29.504932+0000","last_fullsized":"2023-03-29T14:03:29.504932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":47292,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.247914+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:29.247914+0000","last_peered":"2023-03-29T14:03:29.247914+0000","last_clean":"2023-03-29T14:03:29.247914+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:29.247914+0000","last_undegraded":"2023-03-29T14:03:29.247914+0000","last_fullsized":"2023-03-29T14:03:29.247914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"85'31","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.434802+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:30.434802+0000","last_peered":"2023-03-29T14:03:30.434802+0000","last_clean":"2023-03-29T14:03:30.434802+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:30.434802+0000","last_undegraded":"2023-03-29T14:03:30.434802+0000","last_fullsized":"2023-03-29T14:03:30.434802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16571636,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":27261,"num_write":68,"num_write_kb":18202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":144,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.258420+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:29.258420+0000","last_peered":"2023-03-29T14:03:29.258420+0000","last_clean":"2023-03-29T14:03:29.258420+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:29.258420+0000","last_undegraded":"2023-03-29T14:03:29.258420+0000","last_fullsized":"2023-03-29T14:03:29.258420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":20938,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.328834+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:30.328834+0000","last_peered":"2023-03-29T14:03:30.328834+0000","last_clean":"2023-03-29T14:03:30.328834+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:30.328834+0000","last_undegraded":"2023-03-29T14:03:30.328834+0000","last_fullsized":"2023-03-29T14:03:30.328834+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'26","reported_seq":108,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.468156+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:29.468156+0000","last_peered":"2023-03-29T14:03:29.468156+0000","last_clean":"2023-03-29T14:03:29.468156+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:29.468156+0000","last_undegraded":"2023-03-29T14:03:29.468156+0000","last_fullsized":"2023-03-29T14:03:29.468156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5008486,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.256620+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:29.256620+0000","last_peered":"2023-03-29T14:03:29.256620+0000","last_clean":"2023-03-29T14:03:29.256620+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:29.256620+0000","last_undegraded":"2023-03-29T14:03:29.256620+0000","last_fullsized":"2023-03-29T14:03:29.256620+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"80'19","reported_seq":120,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.276332+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:30.276332+0000","last_peered":"2023-03-29T14:03:30.276332+0000","last_clean":"2023-03-29T14:03:30.276332+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:30.276332+0000","last_undegraded":"2023-03-29T14:03:30.276332+0000","last_fullsized":"2023-03-29T14:03:30.276332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857256,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":31039,"num_write":44,"num_write_kb":11186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472179+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:29.472179+0000","last_peered":"2023-03-29T14:03:29.472179+0000","last_clean":"2023-03-29T14:03:29.472179+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:29.472179+0000","last_undegraded":"2023-03-29T14:03:29.472179+0000","last_fullsized":"2023-03-29T14:03:29.472179+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'38","reported_seq":162,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.493086+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:29.493086+0000","last_peered":"2023-03-29T14:03:29.493086+0000","last_clean":"2023-03-29T14:03:29.493086+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:29.493086+0000","last_undegraded":"2023-03-29T14:03:29.493086+0000","last_fullsized":"2023-03-29T14:03:29.493086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9680832,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":79,"num_read_kb":45583,"num_write":80,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.257712+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:29.257712+0000","last_peered":"2023-03-29T14:03:29.257712+0000","last_clean":"2023-03-29T14:03:29.257712+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:29.257712+0000","last_undegraded":"2023-03-29T14:03:29.257712+0000","last_fullsized":"2023-03-29T14:03:29.257712+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":291,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.473707+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:29.473707+0000","last_peered":"2023-03-29T14:03:29.473707+0000","last_clean":"2023-03-29T14:03:29.473707+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:29.473707+0000","last_undegraded":"2023-03-29T14:03:29.473707+0000","last_fullsized":"2023-03-29T14:03:29.473707+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"78'50","reported_seq":196,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.290663+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:30.290663+0000","last_peered":"2023-03-29T14:03:30.290663+0000","last_clean":"2023-03-29T14:03:30.290663+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:30.290663+0000","last_undegraded":"2023-03-29T14:03:30.290663+0000","last_fullsized":"2023-03-29T14:03:30.290663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":51659,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.253720+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:29.253720+0000","last_peered":"2023-03-29T14:03:29.253720+0000","last_clean":"2023-03-29T14:03:29.253720+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:29.253720+0000","last_undegraded":"2023-03-29T14:03:29.253720+0000","last_fullsized":"2023-03-29T14:03:29.253720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"77'96","reported_seq":281,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.478615+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:29.478615+0000","last_peered":"2023-03-29T14:03:29.478615+0000","last_clean":"2023-03-29T14:03:29.478615+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:29.478615+0000","last_undegraded":"2023-03-29T14:03:29.478615+0000","last_fullsized":"2023-03-29T14:03:29.478615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20134786,"num_objects":32,"num_object_clones":20,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":122,"num_read_kb":55629,"num_write":196,"num_write_kb":55794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472005+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:29.472005+0000","last_peered":"2023-03-29T14:03:29.472005+0000","last_clean":"2023-03-29T14:03:29.472005+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:29.472005+0000","last_undegraded":"2023-03-29T14:03:29.472005+0000","last_fullsized":"2023-03-29T14:03:29.472005+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.489162+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:29.489162+0000","last_peered":"2023-03-29T14:03:29.489162+0000","last_clean":"2023-03-29T14:03:29.489162+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:29.489162+0000","last_undegraded":"2023-03-29T14:03:29.489162+0000","last_fullsized":"2023-03-29T14:03:29.489162+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"82'38","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.495026+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:29.495026+0000","last_peered":"2023-03-29T14:03:29.495026+0000","last_clean":"2023-03-29T14:03:29.495026+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:29.495026+0000","last_undegraded":"2023-03-29T14:03:29.495026+0000","last_fullsized":"2023-03-29T14:03:29.495026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25479280,"num_objects":16,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":47,"num_read_kb":18420,"num_write":86,"num_write_kb":25892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.485010+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:29.485010+0000","last_peered":"2023-03-29T14:03:29.485010+0000","last_clean":"2023-03-29T14:03:29.485010+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:29.485010+0000","last_undegraded":"2023-03-29T14:03:29.485010+0000","last_fullsized":"2023-03-29T14:03:29.485010+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.257160+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:29.257160+0000","last_peered":"2023-03-29T14:03:29.257160+0000","last_clean":"2023-03-29T14:03:29.257160+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:29.257160+0000","last_undegraded":"2023-03-29T14:03:29.257160+0000","last_fullsized":"2023-03-29T14:03:29.257160+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"82'41","reported_seq":169,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.460186+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:29.460186+0000","last_peered":"2023-03-29T14:03:29.460186+0000","last_clean":"2023-03-29T14:03:29.460186+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:29.460186+0000","last_undegraded":"2023-03-29T14:03:29.460186+0000","last_fullsized":"2023-03-29T14:03:29.460186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19563282,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":73,"num_read_kb":37554,"num_write":86,"num_write_kb":23154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":208002838,"num_objects":240,"num_object_clones":142,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":926,"num_read_kb":469743,"num_write":1264,"num_write_kb":361632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":598,"ondisk_log_size":598,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":125556,"kb_used_data":118636,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94246284,"statfs":{"total":96636764160,"available":96508194816,"internally_reserved":0,"allocated":121483264,"data_stored":138520611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81168,"kb_used_data":74504,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94290672,"statfs":{"total":96636764160,"available":96553648128,"internally_reserved":0,"allocated":76292096,"data_stored":85245419,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":132248,"kb_used_data":124816,"kb_used_omap":0,"kb_used_meta":7424,"kb_avail":94239592,"statfs":{"total":96636764160,"available":96501342208,"internally_reserved":0,"allocated":127811584,"data_stored":147094833,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7602176},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121776,"kb_used_data":114664,"kb_used_omap":0,"kb_used_meta":7104,"kb_avail":94250064,"statfs":{"total":96636764160,"available":96512065536,"internally_reserved":0,"allocated":117415936,"data_stored":132882186,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7274496},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:33.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:33.587 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:33.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:33.688+0000 7ff526062700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:33.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:33.696+0000 7ff526062700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:33.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:33.734+0000 7f906a1e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:33.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:33.739+0000 7f906a1e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:33.902 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:33.904 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:33.916 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2023-03-29T14:03:33.182027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209183574,"num_objects":244,"num_object_clones":142,"num_object_copies":732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1068,"num_read_kb":469863,"num_write":1538,"num_write_kb":365420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":695,"ondisk_log_size":695,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":460748,"kb_used_data":432620,"kb_used_omap":0,"kb_used_meta":28096,"kb_avail":377026612,"statfs":{"total":386547056640,"available":386075250688,"internally_reserved":0,"allocated":443002880,"data_stored":503743049,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":28770304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5344692,"num_objects":12,"num_object_clones":10,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":41,"num_read_kb":28784,"num_write":42,"num_write_kb":13020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.952688"},"pg_stats":[{"pgid":"3.f","version":"78'21","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472865+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:29.472865+0000","last_peered":"2023-03-29T14:03:29.472865+0000","last_clean":"2023-03-29T14:03:29.472865+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:29.472865+0000","last_undegraded":"2023-03-29T14:03:29.472865+0000","last_fullsized":"2023-03-29T14:03:29.472865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5882450,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":39298,"num_write":42,"num_write_kb":10896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":78,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.320023+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:30.320023+0000","last_peered":"2023-03-29T14:03:30.320023+0000","last_clean":"2023-03-29T14:03:30.320023+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:30.320023+0000","last_undegraded":"2023-03-29T14:03:30.320023+0000","last_fullsized":"2023-03-29T14:03:30.320023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"70'59","reported_seq":191,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.254333+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:29.254333+0000","last_peered":"2023-03-29T14:03:29.254333+0000","last_clean":"2023-03-29T14:03:29.254333+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:29.254333+0000","last_undegraded":"2023-03-29T14:03:29.254333+0000","last_fullsized":"2023-03-29T14:03:29.254333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23428892,"num_objects":18,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":15374,"num_write":124,"num_write_kb":36176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":187,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.255581+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:29.255581+0000","last_peered":"2023-03-29T14:03:29.255581+0000","last_clean":"2023-03-29T14:03:29.255581+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:29.255581+0000","last_undegraded":"2023-03-29T14:03:29.255581+0000","last_fullsized":"2023-03-29T14:03:29.255581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.504932+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:29.504932+0000","last_peered":"2023-03-29T14:03:29.504932+0000","last_clean":"2023-03-29T14:03:29.504932+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:29.504932+0000","last_undegraded":"2023-03-29T14:03:29.504932+0000","last_fullsized":"2023-03-29T14:03:29.504932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":47292,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.247914+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:29.247914+0000","last_peered":"2023-03-29T14:03:29.247914+0000","last_clean":"2023-03-29T14:03:29.247914+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:29.247914+0000","last_undegraded":"2023-03-29T14:03:29.247914+0000","last_fullsized":"2023-03-29T14:03:29.247914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"85'31","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.434802+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:30.434802+0000","last_peered":"2023-03-29T14:03:30.434802+0000","last_clean":"2023-03-29T14:03:30.434802+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:30.434802+0000","last_undegraded":"2023-03-29T14:03:30.434802+0000","last_fullsized":"2023-03-29T14:03:30.434802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16571636,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":27261,"num_write":68,"num_write_kb":18202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":144,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.258420+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:29.258420+0000","last_peered":"2023-03-29T14:03:29.258420+0000","last_clean":"2023-03-29T14:03:29.258420+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:29.258420+0000","last_undegraded":"2023-03-29T14:03:29.258420+0000","last_fullsized":"2023-03-29T14:03:29.258420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":20938,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.328834+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:30.328834+0000","last_peered":"2023-03-29T14:03:30.328834+0000","last_clean":"2023-03-29T14:03:30.328834+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:30.328834+0000","last_undegraded":"2023-03-29T14:03:30.328834+0000","last_fullsized":"2023-03-29T14:03:30.328834+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'26","reported_seq":108,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.468156+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:29.468156+0000","last_peered":"2023-03-29T14:03:29.468156+0000","last_clean":"2023-03-29T14:03:29.468156+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:29.468156+0000","last_undegraded":"2023-03-29T14:03:29.468156+0000","last_fullsized":"2023-03-29T14:03:29.468156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5008486,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.256620+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:29.256620+0000","last_peered":"2023-03-29T14:03:29.256620+0000","last_clean":"2023-03-29T14:03:29.256620+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:29.256620+0000","last_undegraded":"2023-03-29T14:03:29.256620+0000","last_fullsized":"2023-03-29T14:03:29.256620+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"80'19","reported_seq":120,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.276332+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:30.276332+0000","last_peered":"2023-03-29T14:03:30.276332+0000","last_clean":"2023-03-29T14:03:30.276332+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:30.276332+0000","last_undegraded":"2023-03-29T14:03:30.276332+0000","last_fullsized":"2023-03-29T14:03:30.276332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857256,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":31039,"num_write":44,"num_write_kb":11186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472179+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:29.472179+0000","last_peered":"2023-03-29T14:03:29.472179+0000","last_clean":"2023-03-29T14:03:29.472179+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:29.472179+0000","last_undegraded":"2023-03-29T14:03:29.472179+0000","last_fullsized":"2023-03-29T14:03:29.472179+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'38","reported_seq":162,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.493086+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:29.493086+0000","last_peered":"2023-03-29T14:03:29.493086+0000","last_clean":"2023-03-29T14:03:29.493086+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:29.493086+0000","last_undegraded":"2023-03-29T14:03:29.493086+0000","last_fullsized":"2023-03-29T14:03:29.493086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9680832,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":79,"num_read_kb":45583,"num_write":80,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.257712+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:29.257712+0000","last_peered":"2023-03-29T14:03:29.257712+0000","last_clean":"2023-03-29T14:03:29.257712+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:29.257712+0000","last_undegraded":"2023-03-29T14:03:29.257712+0000","last_fullsized":"2023-03-29T14:03:29.257712+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":291,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.473707+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:29.473707+0000","last_peered":"2023-03-29T14:03:29.473707+0000","last_clean":"2023-03-29T14:03:29.473707+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:29.473707+0000","last_undegraded":"2023-03-29T14:03:29.473707+0000","last_fullsized":"2023-03-29T14:03:29.473707+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"78'50","reported_seq":196,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.290663+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:30.290663+0000","last_peered":"2023-03-29T14:03:30.290663+0000","last_clean":"2023-03-29T14:03:30.290663+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:30.290663+0000","last_undegraded":"2023-03-29T14:03:30.290663+0000","last_fullsized":"2023-03-29T14:03:30.290663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":51659,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.253720+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:29.253720+0000","last_peered":"2023-03-29T14:03:29.253720+0000","last_clean":"2023-03-29T14:03:29.253720+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:29.253720+0000","last_undegraded":"2023-03-29T14:03:29.253720+0000","last_fullsized":"2023-03-29T14:03:29.253720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"77'96","reported_seq":281,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.478615+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:29.478615+0000","last_peered":"2023-03-29T14:03:29.478615+0000","last_clean":"2023-03-29T14:03:29.478615+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:29.478615+0000","last_undegraded":"2023-03-29T14:03:29.478615+0000","last_fullsized":"2023-03-29T14:03:29.478615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20134786,"num_objects":32,"num_object_clones":20,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":122,"num_read_kb":55629,"num_write":196,"num_write_kb":55794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472005+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:29.472005+0000","last_peered":"2023-03-29T14:03:29.472005+0000","last_clean":"2023-03-29T14:03:29.472005+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:29.472005+0000","last_undegraded":"2023-03-29T14:03:29.472005+0000","last_fullsized":"2023-03-29T14:03:29.472005+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.489162+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:29.489162+0000","last_peered":"2023-03-29T14:03:29.489162+0000","last_clean":"2023-03-29T14:03:29.489162+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:29.489162+0000","last_undegraded":"2023-03-29T14:03:29.489162+0000","last_fullsized":"2023-03-29T14:03:29.489162+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"82'38","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.495026+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:29.495026+0000","last_peered":"2023-03-29T14:03:29.495026+0000","last_clean":"2023-03-29T14:03:29.495026+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:29.495026+0000","last_undegraded":"2023-03-29T14:03:29.495026+0000","last_fullsized":"2023-03-29T14:03:29.495026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25479280,"num_objects":16,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":47,"num_read_kb":18420,"num_write":86,"num_write_kb":25892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.485010+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:29.485010+0000","last_peered":"2023-03-29T14:03:29.485010+0000","last_clean":"2023-03-29T14:03:29.485010+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:29.485010+0000","last_undegraded":"2023-03-29T14:03:29.485010+0000","last_fullsized":"2023-03-29T14:03:29.485010+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.257160+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:29.257160+0000","last_peered":"2023-03-29T14:03:29.257160+0000","last_clean":"2023-03-29T14:03:29.257160+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:29.257160+0000","last_undegraded":"2023-03-29T14:03:29.257160+0000","last_fullsized":"2023-03-29T14:03:29.257160+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"82'41","reported_seq":169,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.460186+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:29.460186+0000","last_peered":"2023-03-29T14:03:29.460186+0000","last_clean":"2023-03-29T14:03:29.460186+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:29.460186+0000","last_undegraded":"2023-03-29T14:03:29.460186+0000","last_fullsized":"2023-03-29T14:03:29.460186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19563282,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":73,"num_read_kb":37554,"num_write":86,"num_write_kb":23154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":208002838,"num_objects":240,"num_object_clones":142,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":926,"num_read_kb":469743,"num_write":1264,"num_write_kb":361632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":598,"ondisk_log_size":598,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":125556,"kb_used_data":118636,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94246284,"statfs":{"total":96636764160,"available":96508194816,"internally_reserved":0,"allocated":121483264,"data_stored":138520611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81168,"kb_used_data":74504,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94290672,"statfs":{"total":96636764160,"available":96553648128,"internally_reserved":0,"allocated":76292096,"data_stored":85245419,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":132248,"kb_used_data":124816,"kb_used_omap":0,"kb_used_meta":7424,"kb_avail":94239592,"statfs":{"total":96636764160,"available":96501342208,"internally_reserved":0,"allocated":127811584,"data_stored":147094833,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7602176},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121776,"kb_used_data":114664,"kb_used_omap":0,"kb_used_meta":7104,"kb_avail":94250064,"statfs":{"total":96636764160,"available":96512065536,"internally_reserved":0,"allocated":117415936,"data_stored":132882186,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7274496},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:33.921 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:03:33.921 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:33.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:34.049+0000 7f4a5a72f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:34.056 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:34.056+0000 7f4a5a72f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:34.186 INFO:tasks.rados.rados.0.smithi017.stdout:352: done (3 left) 2023-03-29T14:03:34.187 INFO:tasks.rados.rados.0.smithi017.stdout:353: snap_remove snap 36 2023-03-29T14:03:34.259 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:34.261 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:34.272 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-03-29T14:03:34.185704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209183574,"num_objects":244,"num_object_clones":142,"num_object_copies":732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1068,"num_read_kb":469863,"num_write":1538,"num_write_kb":365420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":695,"ondisk_log_size":695,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":460748,"kb_used_data":432620,"kb_used_omap":0,"kb_used_meta":28096,"kb_avail":377026612,"statfs":{"total":386547056640,"available":386075250688,"internally_reserved":0,"allocated":443002880,"data_stored":503743049,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":28770304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5344692,"num_objects":12,"num_object_clones":10,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":41,"num_read_kb":28784,"num_write":42,"num_write_kb":13020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.950429"},"pg_stats":[{"pgid":"3.f","version":"78'21","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472865+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:29.472865+0000","last_peered":"2023-03-29T14:03:29.472865+0000","last_clean":"2023-03-29T14:03:29.472865+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:29.472865+0000","last_undegraded":"2023-03-29T14:03:29.472865+0000","last_fullsized":"2023-03-29T14:03:29.472865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5882450,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":39298,"num_write":42,"num_write_kb":10896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":78,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.320023+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:30.320023+0000","last_peered":"2023-03-29T14:03:30.320023+0000","last_clean":"2023-03-29T14:03:30.320023+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:30.320023+0000","last_undegraded":"2023-03-29T14:03:30.320023+0000","last_fullsized":"2023-03-29T14:03:30.320023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"70'59","reported_seq":191,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.254333+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:29.254333+0000","last_peered":"2023-03-29T14:03:29.254333+0000","last_clean":"2023-03-29T14:03:29.254333+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:29.254333+0000","last_undegraded":"2023-03-29T14:03:29.254333+0000","last_fullsized":"2023-03-29T14:03:29.254333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23428892,"num_objects":18,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":15374,"num_write":124,"num_write_kb":36176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":187,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.255581+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:29.255581+0000","last_peered":"2023-03-29T14:03:29.255581+0000","last_clean":"2023-03-29T14:03:29.255581+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:29.255581+0000","last_undegraded":"2023-03-29T14:03:29.255581+0000","last_fullsized":"2023-03-29T14:03:29.255581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":183,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.504932+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:29.504932+0000","last_peered":"2023-03-29T14:03:29.504932+0000","last_clean":"2023-03-29T14:03:29.504932+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:29.504932+0000","last_undegraded":"2023-03-29T14:03:29.504932+0000","last_fullsized":"2023-03-29T14:03:29.504932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":47292,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.247914+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:29.247914+0000","last_peered":"2023-03-29T14:03:29.247914+0000","last_clean":"2023-03-29T14:03:29.247914+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:29.247914+0000","last_undegraded":"2023-03-29T14:03:29.247914+0000","last_fullsized":"2023-03-29T14:03:29.247914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"85'31","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.434802+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:30.434802+0000","last_peered":"2023-03-29T14:03:30.434802+0000","last_clean":"2023-03-29T14:03:30.434802+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:30.434802+0000","last_undegraded":"2023-03-29T14:03:30.434802+0000","last_fullsized":"2023-03-29T14:03:30.434802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16571636,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":27261,"num_write":68,"num_write_kb":18202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":144,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.258420+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:29.258420+0000","last_peered":"2023-03-29T14:03:29.258420+0000","last_clean":"2023-03-29T14:03:29.258420+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:29.258420+0000","last_undegraded":"2023-03-29T14:03:29.258420+0000","last_fullsized":"2023-03-29T14:03:29.258420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":20938,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.328834+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:30.328834+0000","last_peered":"2023-03-29T14:03:30.328834+0000","last_clean":"2023-03-29T14:03:30.328834+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:30.328834+0000","last_undegraded":"2023-03-29T14:03:30.328834+0000","last_fullsized":"2023-03-29T14:03:30.328834+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'26","reported_seq":108,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.468156+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:29.468156+0000","last_peered":"2023-03-29T14:03:29.468156+0000","last_clean":"2023-03-29T14:03:29.468156+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:29.468156+0000","last_undegraded":"2023-03-29T14:03:29.468156+0000","last_fullsized":"2023-03-29T14:03:29.468156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5008486,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.256620+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:29.256620+0000","last_peered":"2023-03-29T14:03:29.256620+0000","last_clean":"2023-03-29T14:03:29.256620+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:29.256620+0000","last_undegraded":"2023-03-29T14:03:29.256620+0000","last_fullsized":"2023-03-29T14:03:29.256620+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"80'19","reported_seq":120,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.276332+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:30.276332+0000","last_peered":"2023-03-29T14:03:30.276332+0000","last_clean":"2023-03-29T14:03:30.276332+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:30.276332+0000","last_undegraded":"2023-03-29T14:03:30.276332+0000","last_fullsized":"2023-03-29T14:03:30.276332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857256,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":31039,"num_write":44,"num_write_kb":11186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472179+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:29.472179+0000","last_peered":"2023-03-29T14:03:29.472179+0000","last_clean":"2023-03-29T14:03:29.472179+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:29.472179+0000","last_undegraded":"2023-03-29T14:03:29.472179+0000","last_fullsized":"2023-03-29T14:03:29.472179+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'38","reported_seq":162,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.493086+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:29.493086+0000","last_peered":"2023-03-29T14:03:29.493086+0000","last_clean":"2023-03-29T14:03:29.493086+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:29.493086+0000","last_undegraded":"2023-03-29T14:03:29.493086+0000","last_fullsized":"2023-03-29T14:03:29.493086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9680832,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":79,"num_read_kb":45583,"num_write":80,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.257712+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:29.257712+0000","last_peered":"2023-03-29T14:03:29.257712+0000","last_clean":"2023-03-29T14:03:29.257712+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:29.257712+0000","last_undegraded":"2023-03-29T14:03:29.257712+0000","last_fullsized":"2023-03-29T14:03:29.257712+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":291,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.473707+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:29.473707+0000","last_peered":"2023-03-29T14:03:29.473707+0000","last_clean":"2023-03-29T14:03:29.473707+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:29.473707+0000","last_undegraded":"2023-03-29T14:03:29.473707+0000","last_fullsized":"2023-03-29T14:03:29.473707+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"78'50","reported_seq":196,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:30.290663+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:30.290663+0000","last_peered":"2023-03-29T14:03:30.290663+0000","last_clean":"2023-03-29T14:03:30.290663+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:30.290663+0000","last_undegraded":"2023-03-29T14:03:30.290663+0000","last_fullsized":"2023-03-29T14:03:30.290663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":51659,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.253720+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:29.253720+0000","last_peered":"2023-03-29T14:03:29.253720+0000","last_clean":"2023-03-29T14:03:29.253720+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:29.253720+0000","last_undegraded":"2023-03-29T14:03:29.253720+0000","last_fullsized":"2023-03-29T14:03:29.253720+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"77'96","reported_seq":281,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.478615+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:29.478615+0000","last_peered":"2023-03-29T14:03:29.478615+0000","last_clean":"2023-03-29T14:03:29.478615+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:29.478615+0000","last_undegraded":"2023-03-29T14:03:29.478615+0000","last_fullsized":"2023-03-29T14:03:29.478615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20134786,"num_objects":32,"num_object_clones":20,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":122,"num_read_kb":55629,"num_write":196,"num_write_kb":55794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.472005+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:29.472005+0000","last_peered":"2023-03-29T14:03:29.472005+0000","last_clean":"2023-03-29T14:03:29.472005+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:29.472005+0000","last_undegraded":"2023-03-29T14:03:29.472005+0000","last_fullsized":"2023-03-29T14:03:29.472005+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.489162+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:29.489162+0000","last_peered":"2023-03-29T14:03:29.489162+0000","last_clean":"2023-03-29T14:03:29.489162+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:29.489162+0000","last_undegraded":"2023-03-29T14:03:29.489162+0000","last_fullsized":"2023-03-29T14:03:29.489162+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"82'38","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.495026+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:29.495026+0000","last_peered":"2023-03-29T14:03:29.495026+0000","last_clean":"2023-03-29T14:03:29.495026+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:29.495026+0000","last_undegraded":"2023-03-29T14:03:29.495026+0000","last_fullsized":"2023-03-29T14:03:29.495026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25479280,"num_objects":16,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":47,"num_read_kb":18420,"num_write":86,"num_write_kb":25892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.485010+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:29.485010+0000","last_peered":"2023-03-29T14:03:29.485010+0000","last_clean":"2023-03-29T14:03:29.485010+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:29.485010+0000","last_undegraded":"2023-03-29T14:03:29.485010+0000","last_fullsized":"2023-03-29T14:03:29.485010+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.257160+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:29.257160+0000","last_peered":"2023-03-29T14:03:29.257160+0000","last_clean":"2023-03-29T14:03:29.257160+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:29.257160+0000","last_undegraded":"2023-03-29T14:03:29.257160+0000","last_fullsized":"2023-03-29T14:03:29.257160+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"82'41","reported_seq":169,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-03-29T14:03:29.460186+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:29.460186+0000","last_peered":"2023-03-29T14:03:29.460186+0000","last_clean":"2023-03-29T14:03:29.460186+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:29.460186+0000","last_undegraded":"2023-03-29T14:03:29.460186+0000","last_fullsized":"2023-03-29T14:03:29.460186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19563282,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":73,"num_read_kb":37554,"num_write":86,"num_write_kb":23154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":208002838,"num_objects":240,"num_object_clones":142,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":926,"num_read_kb":469743,"num_write":1264,"num_write_kb":361632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":598,"ondisk_log_size":598,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":125556,"kb_used_data":118636,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94246284,"statfs":{"total":96636764160,"available":96508194816,"internally_reserved":0,"allocated":121483264,"data_stored":138520611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":81168,"kb_used_data":74504,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94290672,"statfs":{"total":96636764160,"available":96553648128,"internally_reserved":0,"allocated":76292096,"data_stored":85245419,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":132248,"kb_used_data":124816,"kb_used_omap":0,"kb_used_meta":7424,"kb_avail":94239592,"statfs":{"total":96636764160,"available":96501342208,"internally_reserved":0,"allocated":127811584,"data_stored":147094833,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7602176},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":121776,"kb_used_data":114664,"kb_used_omap":0,"kb_used_meta":7104,"kb_avail":94250064,"statfs":{"total":96636764160,"available":96512065536,"internally_reserved":0,"allocated":117415936,"data_stored":132882186,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7274496},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:35.188 INFO:tasks.rados.rados.0.smithi017.stdout:353: done (3 left) 2023-03-29T14:03:35.189 INFO:tasks.rados.rados.0.smithi017.stdout:354: delete oid 15 current snap is 38 2023-03-29T14:03:35.189 INFO:tasks.rados.rados.0.smithi017.stdout:350: expect (ObjNum 132 snap 15 seq_num 132) 2023-03-29T14:03:35.276 INFO:tasks.rados.rados.0.smithi017.stdout:349: expect (ObjNum 184 snap 34 seq_num 184) 2023-03-29T14:03:35.468 INFO:tasks.rados.rados.0.smithi017.stdout:348: expect (ObjNum 142 snap 18 seq_num 142) 2023-03-29T14:03:35.562 INFO:tasks.rados.rados.0.smithi017.stdout:348: done (3 left) 2023-03-29T14:03:35.562 INFO:tasks.rados.rados.0.smithi017.stdout:349: done (2 left) 2023-03-29T14:03:35.563 INFO:tasks.rados.rados.0.smithi017.stdout:350: done (1 left) 2023-03-29T14:03:35.563 INFO:tasks.rados.rados.0.smithi017.stdout:354: done (0 left) 2023-03-29T14:03:35.563 INFO:tasks.rados.rados.0.smithi017.stdout:355: write (excl) oid 24 current snap is 38 2023-03-29T14:03:35.563 INFO:tasks.rados.rados.0.smithi017.stdout:355: seq_num 190 ranges {793330=689782,2035461=617731,3101545=91785} 2023-03-29T14:03:35.570 INFO:tasks.rados.rados.0.smithi017.stdout:355: writing smithi017107884-24 from 793330 to 1483112 tid 1 2023-03-29T14:03:35.577 INFO:tasks.rados.rados.0.smithi017.stdout:355: writing smithi017107884-24 from 2035461 to 2653192 tid 2 2023-03-29T14:03:35.580 INFO:tasks.rados.rados.0.smithi017.stdout:355: writing smithi017107884-24 from 3101545 to 3193330 tid 3 2023-03-29T14:03:35.580 INFO:tasks.rados.rados.0.smithi017.stdout:356: read oid 7 snap -1 2023-03-29T14:03:35.580 INFO:tasks.rados.rados.0.smithi017.stdout:356: expect (ObjNum 166 snap 26 seq_num 166) 2023-03-29T14:03:35.580 INFO:tasks.rados.rados.0.smithi017.stdout:357: read oid 14 snap -1 2023-03-29T14:03:35.580 INFO:tasks.rados.rados.0.smithi017.stdout:357: expect (ObjNum 187 snap 36 seq_num 187) 2023-03-29T14:03:35.581 INFO:tasks.rados.rados.0.smithi017.stdout:358: write oid 36 current snap is 38 2023-03-29T14:03:35.581 INFO:tasks.rados.rados.0.smithi017.stdout:358: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2023-03-29T14:03:35.585 INFO:tasks.rados.rados.0.smithi017.stdout:358: writing smithi017107884-36 from 410137 to 891521 tid 1 2023-03-29T14:03:35.592 INFO:tasks.rados.rados.0.smithi017.stdout:358: writing smithi017107884-36 from 1410296 to 2171685 tid 2 2023-03-29T14:03:35.599 INFO:tasks.rados.rados.0.smithi017.stdout:358: writing smithi017107884-36 from 2728968 to 3210137 tid 3 2023-03-29T14:03:35.599 INFO:tasks.rados.rados.0.smithi017.stdout:359: read oid 4 snap -1 2023-03-29T14:03:35.599 INFO:tasks.rados.rados.0.smithi017.stdout:359: expect (ObjNum 117 snap 10 seq_num 117) 2023-03-29T14:03:35.599 INFO:tasks.rados.rados.0.smithi017.stdout:360: read oid 45 snap -1 2023-03-29T14:03:35.599 INFO:tasks.rados.rados.0.smithi017.stdout:360: expect (ObjNum 151 snap 21 seq_num 151) 2023-03-29T14:03:35.599 INFO:tasks.rados.rados.0.smithi017.stdout:361: write oid 47 current snap is 38 2023-03-29T14:03:35.599 INFO:tasks.rados.rados.0.smithi017.stdout:361: seq_num 192 ranges {426944=556633,1468778=621400,2872744=354200} 2023-03-29T14:03:35.604 INFO:tasks.rados.rados.0.smithi017.stdout:361: writing smithi017107884-47 from 426944 to 983577 tid 1 2023-03-29T14:03:35.611 INFO:tasks.rados.rados.0.smithi017.stdout:361: writing smithi017107884-47 from 1468778 to 2090178 tid 2 2023-03-29T14:03:35.617 INFO:tasks.rados.rados.0.smithi017.stdout:361: writing smithi017107884-47 from 2872744 to 3226944 tid 3 2023-03-29T14:03:35.617 INFO:tasks.rados.rados.0.smithi017.stdout:362: snap_create 2023-03-29T14:03:35.617 INFO:tasks.rados.rados.0.smithi017.stdout:356: expect (ObjNum 166 snap 26 seq_num 166) 2023-03-29T14:03:35.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:35.840 INFO:tasks.rados.rados.0.smithi017.stdout:355: finishing write tid 1 to smithi017107884-24 2023-03-29T14:03:35.840 INFO:tasks.rados.rados.0.smithi017.stdout:357: expect (ObjNum 187 snap 36 seq_num 187) 2023-03-29T14:03:35.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:35.939+0000 7f1fe32f5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:35.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:35.944+0000 7f1fe32f5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:36.030 INFO:tasks.rados.rados.0.smithi017.stdout:358: finishing write tid 1 to smithi017107884-36 2023-03-29T14:03:36.031 INFO:tasks.rados.rados.0.smithi017.stdout:359: expect (ObjNum 117 snap 10 seq_num 117) 2023-03-29T14:03:36.286 INFO:tasks.rados.rados.0.smithi017.stdout:355: finishing write tid 2 to smithi017107884-24 2023-03-29T14:03:36.286 INFO:tasks.rados.rados.0.smithi017.stdout:355: finishing write tid 3 to smithi017107884-24 2023-03-29T14:03:36.286 INFO:tasks.rados.rados.0.smithi017.stdout:355: finishing write tid 4 to smithi017107884-24 2023-03-29T14:03:36.286 INFO:tasks.rados.rados.0.smithi017.stdout:355: finishing write tid 5 to smithi017107884-24 2023-03-29T14:03:36.286 INFO:tasks.rados.rados.0.smithi017.stdout:355: oid 24 updating version 0 to 24 2023-03-29T14:03:36.286 INFO:tasks.rados.rados.0.smithi017.stdout:355: oid 24 version 24 is already newer than 24 2023-03-29T14:03:36.286 INFO:tasks.rados.rados.0.smithi017.stdout:355: oid 24 version 24 is already newer than 24 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:355: oid 24 version 24 is already newer than 20 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 24 (ObjNum 190 snap 38 seq_num 190) dirty exists 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:355: left oid 24 (ObjNum 190 snap 38 seq_num 190) 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:358: finishing write tid 2 to smithi017107884-36 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:358: finishing write tid 3 to smithi017107884-36 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:355: done (7 left) 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:356: done (6 left) 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:357: done (5 left) 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:359: done (4 left) 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:362: done (3 left) 2023-03-29T14:03:36.287 INFO:tasks.rados.rados.0.smithi017.stdout:363: snap_remove snap 34 2023-03-29T14:03:37.257 INFO:tasks.rados.rados.0.smithi017.stdout:363: done (3 left) 2023-03-29T14:03:37.257 INFO:tasks.rados.rados.0.smithi017.stdout:364: snap_create 2023-03-29T14:03:37.257 INFO:tasks.rados.rados.0.smithi017.stdout:358: finishing write tid 4 to smithi017107884-36 2023-03-29T14:03:37.257 INFO:tasks.rados.rados.0.smithi017.stdout:360: expect (ObjNum 151 snap 21 seq_num 151) 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:358: finishing write tid 5 to smithi017107884-36 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:358: oid 36 updating version 0 to 43 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:358: oid 36 version 43 is already newer than 43 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:358: oid 36 version 43 is already newer than 43 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:358: oid 36 version 43 is already newer than 41 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 43 (ObjNum 191 snap 38 seq_num 191) dirty exists 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:358: left oid 36 (ObjNum 191 snap 38 seq_num 191) 2023-03-29T14:03:37.436 INFO:tasks.rados.rados.0.smithi017.stdout:361: finishing write tid 1 to smithi017107884-47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: finishing write tid 2 to smithi017107884-47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: finishing write tid 3 to smithi017107884-47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: finishing write tid 4 to smithi017107884-47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: finishing write tid 5 to smithi017107884-47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: oid 47 updating version 0 to 47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: oid 47 version 47 is already newer than 47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: oid 47 version 47 is already newer than 47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: oid 47 version 47 is already newer than 47 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 47 (ObjNum 192 snap 38 seq_num 192) dirty exists 2023-03-29T14:03:37.437 INFO:tasks.rados.rados.0.smithi017.stdout:361: left oid 47 (ObjNum 192 snap 38 seq_num 192) 2023-03-29T14:03:38.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:38.146 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:38.145+0000 7f147b000700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:38.149 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:38.149+0000 7f1479d9e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:38.260 INFO:tasks.rados.rados.0.smithi017.stdout:358: done (3 left) 2023-03-29T14:03:38.260 INFO:tasks.rados.rados.0.smithi017.stdout:360: done (2 left) 2023-03-29T14:03:38.261 INFO:tasks.rados.rados.0.smithi017.stdout:361: done (1 left) 2023-03-29T14:03:38.261 INFO:tasks.rados.rados.0.smithi017.stdout:364: done (0 left) 2023-03-29T14:03:38.261 INFO:tasks.rados.rados.0.smithi017.stdout:365: write (excl) oid 4 current snap is 40 2023-03-29T14:03:38.261 INFO:tasks.rados.rados.0.smithi017.stdout:365: seq_num 193 ranges {443751=631882,1810907=765058,3067461=176290} 2023-03-29T14:03:38.266 INFO:tasks.rados.rados.0.smithi017.stdout:365: writing smithi017107884-4 from 443751 to 1075633 tid 1 2023-03-29T14:03:38.274 INFO:tasks.rados.rados.0.smithi017.stdout:365: writing smithi017107884-4 from 1810907 to 2575965 tid 2 2023-03-29T14:03:38.278 INFO:tasks.rados.rados.0.smithi017.stdout:365: writing smithi017107884-4 from 3067461 to 3243751 tid 3 2023-03-29T14:03:38.278 INFO:tasks.rados.rados.0.smithi017.stdout:366: write (excl) oid 22 current snap is 40 2023-03-29T14:03:38.278 INFO:tasks.rados.rados.0.smithi017.stdout:366: seq_num 194 ranges {460558=707131,1869389=508716,3094884=165674} 2023-03-29T14:03:38.284 INFO:tasks.rados.rados.0.smithi017.stdout:366: writing smithi017107884-22 from 460558 to 1167689 tid 1 2023-03-29T14:03:38.291 INFO:tasks.rados.rados.0.smithi017.stdout:366: writing smithi017107884-22 from 1869389 to 2378105 tid 2 2023-03-29T14:03:38.296 INFO:tasks.rados.rados.0.smithi017.stdout:366: writing smithi017107884-22 from 3094884 to 3260558 tid 3 2023-03-29T14:03:38.296 INFO:tasks.rados.rados.0.smithi017.stdout:367: write oid 34 current snap is 40 2023-03-29T14:03:38.296 INFO:tasks.rados.rados.0.smithi017.stdout:367: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2023-03-29T14:03:38.304 INFO:tasks.rados.rados.0.smithi017.stdout:367: writing smithi017107884-34 from 477365 to 1259745 tid 1 2023-03-29T14:03:38.311 INFO:tasks.rados.rados.0.smithi017.stdout:367: writing smithi017107884-34 from 1927871 to 2696598 tid 2 2023-03-29T14:03:38.314 INFO:tasks.rados.rados.0.smithi017.stdout:367: writing smithi017107884-34 from 3122307 to 3277365 tid 3 2023-03-29T14:03:38.314 INFO:tasks.rados.rados.0.smithi017.stdout:368: write oid 15 current snap is 40 2023-03-29T14:03:38.314 INFO:tasks.rados.rados.0.smithi017.stdout:368: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2023-03-29T14:03:38.319 INFO:tasks.rados.rados.0.smithi017.stdout:368: writing smithi017107884-15 from 494172 to 951801 tid 1 2023-03-29T14:03:38.326 INFO:tasks.rados.rados.0.smithi017.stdout:368: writing smithi017107884-15 from 1586353 to 2098738 tid 2 2023-03-29T14:03:38.332 INFO:tasks.rados.rados.0.smithi017.stdout:368: writing smithi017107884-15 from 2749730 to 3294172 tid 3 2023-03-29T14:03:38.332 INFO:tasks.rados.rados.0.smithi017.stdout:369: read oid 10 snap 38 2023-03-29T14:03:38.333 INFO:tasks.rados.rados.0.smithi017.stdout:369: expect (ObjNum 146 snap 20 seq_num 146) 2023-03-29T14:03:38.333 INFO:tasks.rados.rados.0.smithi017.stdout:370: write (excl) oid 12 current snap is 40 2023-03-29T14:03:38.333 INFO:tasks.rados.rados.0.smithi017.stdout:370: seq_num 197 ranges {510979=532878,1528482=772396,3060800=250179} 2023-03-29T14:03:38.338 INFO:tasks.rados.rados.0.smithi017.stdout:370: writing smithi017107884-12 from 510979 to 1043857 tid 1 2023-03-29T14:03:38.345 INFO:tasks.rados.rados.0.smithi017.stdout:370: writing smithi017107884-12 from 1528482 to 2300878 tid 2 2023-03-29T14:03:38.351 INFO:tasks.rados.rados.0.smithi017.stdout:370: writing smithi017107884-12 from 3060800 to 3310979 tid 3 2023-03-29T14:03:38.351 INFO:tasks.rados.rados.0.smithi017.stdout:371: read oid 37 snap 38 2023-03-29T14:03:38.351 INFO:tasks.rados.rados.0.smithi017.stdout:371: expect deleted 2023-03-29T14:03:38.351 INFO:tasks.rados.rados.0.smithi017.stdout:372: write (excl) oid 19 current snap is 40 2023-03-29T14:03:38.351 INFO:tasks.rados.rados.0.smithi017.stdout:372: seq_num 198 ranges {527786=724480,1703317=516054,2804576=435035} 2023-03-29T14:03:38.359 INFO:tasks.rados.rados.0.smithi017.stdout:372: writing smithi017107884-19 from 527786 to 1252266 tid 1 2023-03-29T14:03:38.365 INFO:tasks.rados.rados.0.smithi017.stdout:372: writing smithi017107884-19 from 1703317 to 2219371 tid 2 2023-03-29T14:03:38.371 INFO:tasks.rados.rados.0.smithi017.stdout:372: writing smithi017107884-19 from 2804576 to 3239611 tid 3 2023-03-29T14:03:38.371 INFO:tasks.rados.rados.0.smithi017.stdout:373: snap_create 2023-03-29T14:03:38.410 INFO:tasks.rados.rados.0.smithi017.stdout:368: finishing write tid 1 to smithi017107884-15 2023-03-29T14:03:38.441 INFO:tasks.rados.rados.0.smithi017.stdout:366: finishing write tid 1 to smithi017107884-22 2023-03-29T14:03:38.446 INFO:tasks.rados.rados.0.smithi017.stdout:365: finishing write tid 1 to smithi017107884-4 2023-03-29T14:03:38.449 INFO:tasks.rados.rados.0.smithi017.stdout:368: finishing write tid 2 to smithi017107884-15 2023-03-29T14:03:38.473 INFO:tasks.rados.rados.0.smithi017.stdout:368: finishing write tid 3 to smithi017107884-15 2023-03-29T14:03:38.494 INFO:tasks.rados.rados.0.smithi017.stdout:368: finishing write tid 4 to smithi017107884-15 2023-03-29T14:03:38.502 INFO:tasks.rados.rados.0.smithi017.stdout:368: finishing write tid 5 to smithi017107884-15 2023-03-29T14:03:38.502 INFO:tasks.rados.rados.0.smithi017.stdout:368: oid 15 updating version 0 to 102 2023-03-29T14:03:38.502 INFO:tasks.rados.rados.0.smithi017.stdout:368: oid 15 version 102 is already newer than 102 2023-03-29T14:03:38.503 INFO:tasks.rados.rados.0.smithi017.stdout:368: oid 15 version 102 is already newer than 102 2023-03-29T14:03:38.503 INFO:tasks.rados.rados.0.smithi017.stdout:368: oid 15 version 102 is already newer than 102 2023-03-29T14:03:38.503 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 102 (ObjNum 196 snap 40 seq_num 196) dirty exists 2023-03-29T14:03:38.503 INFO:tasks.rados.rados.0.smithi017.stdout:368: left oid 15 (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:03:38.531 INFO:tasks.rados.rados.0.smithi017.stdout:367: finishing write tid 1 to smithi017107884-34 2023-03-29T14:03:38.540 INFO:tasks.rados.rados.0.smithi017.stdout:372: finishing write tid 1 to smithi017107884-19 2023-03-29T14:03:38.559 INFO:tasks.rados.rados.0.smithi017.stdout:366: finishing write tid 2 to smithi017107884-22 2023-03-29T14:03:38.560 INFO:tasks.rados.rados.0.smithi017.stdout:367: finishing write tid 2 to smithi017107884-34 2023-03-29T14:03:38.581 INFO:tasks.rados.rados.0.smithi017.stdout:367: finishing write tid 3 to smithi017107884-34 2023-03-29T14:03:38.583 INFO:tasks.rados.rados.0.smithi017.stdout:366: finishing write tid 3 to smithi017107884-22 2023-03-29T14:03:38.585 INFO:tasks.rados.rados.0.smithi017.stdout:367: finishing write tid 4 to smithi017107884-34 2023-03-29T14:03:38.589 INFO:tasks.rados.rados.0.smithi017.stdout:366: finishing write tid 4 to smithi017107884-22 2023-03-29T14:03:38.590 INFO:tasks.rados.rados.0.smithi017.stdout:366: finishing write tid 5 to smithi017107884-22 2023-03-29T14:03:38.590 INFO:tasks.rados.rados.0.smithi017.stdout:366: oid 22 updating version 0 to 22 2023-03-29T14:03:38.590 INFO:tasks.rados.rados.0.smithi017.stdout:366: oid 22 updating version 22 to 25 2023-03-29T14:03:38.590 INFO:tasks.rados.rados.0.smithi017.stdout:366: oid 22 version 25 is already newer than 25 2023-03-29T14:03:38.590 INFO:tasks.rados.rados.0.smithi017.stdout:366: oid 22 version 25 is already newer than 25 2023-03-29T14:03:38.590 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 25 (ObjNum 194 snap 40 seq_num 194) dirty exists 2023-03-29T14:03:38.590 INFO:tasks.rados.rados.0.smithi017.stdout:366: left oid 22 (ObjNum 194 snap 40 seq_num 194) 2023-03-29T14:03:38.593 INFO:tasks.rados.rados.0.smithi017.stdout:365: finishing write tid 2 to smithi017107884-4 2023-03-29T14:03:38.593 INFO:tasks.rados.rados.0.smithi017.stdout:365: finishing write tid 3 to smithi017107884-4 2023-03-29T14:03:38.594 INFO:tasks.rados.rados.0.smithi017.stdout:365: finishing write tid 4 to smithi017107884-4 2023-03-29T14:03:38.596 INFO:tasks.rados.rados.0.smithi017.stdout:365: finishing write tid 5 to smithi017107884-4 2023-03-29T14:03:38.596 INFO:tasks.rados.rados.0.smithi017.stdout:365: oid 4 updating version 0 to 64 2023-03-29T14:03:38.596 INFO:tasks.rados.rados.0.smithi017.stdout:365: oid 4 version 64 is already newer than 64 2023-03-29T14:03:38.596 INFO:tasks.rados.rados.0.smithi017.stdout:365: oid 4 version 64 is already newer than 60 2023-03-29T14:03:38.596 INFO:tasks.rados.rados.0.smithi017.stdout:365: oid 4 version 64 is already newer than 64 2023-03-29T14:03:38.596 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 64 (ObjNum 193 snap 40 seq_num 193) dirty exists 2023-03-29T14:03:38.596 INFO:tasks.rados.rados.0.smithi017.stdout:365: left oid 4 (ObjNum 193 snap 40 seq_num 193) 2023-03-29T14:03:38.597 INFO:tasks.rados.rados.0.smithi017.stdout:367: finishing write tid 5 to smithi017107884-34 2023-03-29T14:03:38.598 INFO:tasks.rados.rados.0.smithi017.stdout:367: oid 34 updating version 0 to 55 2023-03-29T14:03:38.598 INFO:tasks.rados.rados.0.smithi017.stdout:367: oid 34 version 55 is already newer than 55 2023-03-29T14:03:38.598 INFO:tasks.rados.rados.0.smithi017.stdout:367: oid 34 version 55 is already newer than 55 2023-03-29T14:03:38.598 INFO:tasks.rados.rados.0.smithi017.stdout:367: oid 34 version 55 is already newer than 55 2023-03-29T14:03:38.598 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 55 (ObjNum 195 snap 40 seq_num 195) dirty exists 2023-03-29T14:03:38.598 INFO:tasks.rados.rados.0.smithi017.stdout:367: left oid 34 (ObjNum 195 snap 40 seq_num 195) 2023-03-29T14:03:38.605 INFO:tasks.rados.rados.0.smithi017.stdout:372: finishing write tid 2 to smithi017107884-19 2023-03-29T14:03:38.609 INFO:tasks.rados.rados.0.smithi017.stdout:372: finishing write tid 3 to smithi017107884-19 2023-03-29T14:03:38.613 INFO:tasks.rados.rados.0.smithi017.stdout:370: finishing write tid 1 to smithi017107884-12 2023-03-29T14:03:38.616 INFO:tasks.rados.rados.0.smithi017.stdout:372: finishing write tid 4 to smithi017107884-19 2023-03-29T14:03:38.617 INFO:tasks.rados.rados.0.smithi017.stdout:369: expect (ObjNum 146 snap 20 seq_num 146) 2023-03-29T14:03:38.711 INFO:tasks.rados.rados.0.smithi017.stdout:372: finishing write tid 5 to smithi017107884-19 2023-03-29T14:03:38.712 INFO:tasks.rados.rados.0.smithi017.stdout:372: oid 19 updating version 0 to 46 2023-03-29T14:03:38.712 INFO:tasks.rados.rados.0.smithi017.stdout:372: oid 19 version 46 is already newer than 42 2023-03-29T14:03:38.712 INFO:tasks.rados.rados.0.smithi017.stdout:372: oid 19 version 46 is already newer than 46 2023-03-29T14:03:38.712 INFO:tasks.rados.rados.0.smithi017.stdout:372: oid 19 version 46 is already newer than 46 2023-03-29T14:03:38.712 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 46 (ObjNum 198 snap 40 seq_num 198) dirty exists 2023-03-29T14:03:38.712 INFO:tasks.rados.rados.0.smithi017.stdout:372: left oid 19 (ObjNum 198 snap 40 seq_num 198) 2023-03-29T14:03:38.712 INFO:tasks.rados.rados.0.smithi017.stdout:370: finishing write tid 2 to smithi017107884-12 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: finishing write tid 3 to smithi017107884-12 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: finishing write tid 4 to smithi017107884-12 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: finishing write tid 5 to smithi017107884-12 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: oid 12 updating version 0 to 52 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: oid 12 version 52 is already newer than 52 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: oid 12 version 52 is already newer than 52 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: oid 12 version 52 is already newer than 48 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 52 (ObjNum 197 snap 40 seq_num 197) dirty exists 2023-03-29T14:03:38.713 INFO:tasks.rados.rados.0.smithi017.stdout:370: left oid 12 (ObjNum 197 snap 40 seq_num 197) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:365: done (8 left) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:366: done (7 left) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:367: done (6 left) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:368: done (5 left) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:369: done (4 left) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:370: done (3 left) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:371: done (2 left) 2023-03-29T14:03:39.264 INFO:tasks.rados.rados.0.smithi017.stdout:372: done (1 left) 2023-03-29T14:03:39.265 INFO:tasks.rados.rados.0.smithi017.stdout:373: done (0 left) 2023-03-29T14:03:39.265 INFO:tasks.rados.rados.0.smithi017.stdout:374: write oid 33 current snap is 41 2023-03-29T14:03:39.265 INFO:tasks.rados.rados.0.smithi017.stdout:374: seq_num 199 ranges {544593=799729,1761799=776065,2948352=396241} 2023-03-29T14:03:39.271 INFO:tasks.rados.rados.0.smithi017.stdout:374: writing smithi017107884-33 from 544593 to 1344322 tid 1 2023-03-29T14:03:39.277 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:39.277 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:39.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:39.278 INFO:tasks.rados.rados.0.smithi017.stdout:374: writing smithi017107884-33 from 1761799 to 2537864 tid 2 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:374: writing smithi017107884-33 from 2948352 to 3344593 tid 3 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:375: read oid 5 snap -1 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:375: expect deleted 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:376: read oid 32 snap -1 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:376: expect (ObjNum 173 snap 29 seq_num 173) 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:377: read oid 38 snap -1 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:377: expect (ObjNum 132 snap 15 seq_num 132) 2023-03-29T14:03:39.283 INFO:tasks.rados.rados.0.smithi017.stdout:378: snap_create 2023-03-29T14:03:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:39.415+0000 7f4819d1a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:39.422+0000 7f4819d1a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:39.428 INFO:tasks.rados.rados.0.smithi017.stdout:374: finishing write tid 1 to smithi017107884-33 2023-03-29T14:03:39.447 INFO:tasks.rados.rados.0.smithi017.stdout:376: expect (ObjNum 173 snap 29 seq_num 173) 2023-03-29T14:03:39.562 INFO:tasks.rados.rados.0.smithi017.stdout:377: expect (ObjNum 132 snap 15 seq_num 132) 2023-03-29T14:03:39.639 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:39.641 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: finishing write tid 2 to smithi017107884-33 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: finishing write tid 3 to smithi017107884-33 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: finishing write tid 4 to smithi017107884-33 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: finishing write tid 5 to smithi017107884-33 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: oid 33 updating version 0 to 51 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: oid 33 version 51 is already newer than 51 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: oid 33 version 51 is already newer than 51 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: oid 33 version 51 is already newer than 51 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 51 (ObjNum 199 snap 41 seq_num 199) dirty exists 2023-03-29T14:03:39.648 INFO:tasks.rados.rados.0.smithi017.stdout:374: left oid 33 (ObjNum 199 snap 41 seq_num 199) 2023-03-29T14:03:39.653 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2023-03-29T14:03:39.263436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209755144,"num_objects":254,"num_object_clones":152,"num_object_copies":762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1125,"num_read_kb":517709,"num_write":1576,"num_write_kb":376774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":474848,"kb_used_data":446464,"kb_used_omap":0,"kb_used_meta":28352,"kb_avail":377012512,"statfs":{"total":386547056640,"available":386060812288,"internally_reserved":0,"allocated":457179136,"data_stored":524599182,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":29032448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":571570,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":47846,"num_write":38,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.076334"},"pg_stats":[{"pgid":"3.f","version":"78'21","reported_seq":117,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-03-29T14:03:33.230343+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:33.230343+0000","last_peered":"2023-03-29T14:03:33.230343+0000","last_clean":"2023-03-29T14:03:33.230343+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:33.230343+0000","last_undegraded":"2023-03-29T14:03:33.230343+0000","last_fullsized":"2023-03-29T14:03:33.230343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5882450,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":39298,"num_write":42,"num_write_kb":10896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.317729+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:35.317729+0000","last_peered":"2023-03-29T14:03:35.317729+0000","last_clean":"2023-03-29T14:03:35.317729+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:35.317729+0000","last_undegraded":"2023-03-29T14:03:35.317729+0000","last_fullsized":"2023-03-29T14:03:35.317729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":5383,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"70'59","reported_seq":197,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.646015+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:35.646015+0000","last_peered":"2023-03-29T14:03:35.646015+0000","last_clean":"2023-03-29T14:03:35.646015+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:35.646015+0000","last_undegraded":"2023-03-29T14:03:35.646015+0000","last_fullsized":"2023-03-29T14:03:35.646015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23428892,"num_objects":18,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":59,"num_read_kb":25344,"num_write":124,"num_write_kb":36176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":190,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:34.223432+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:34.223432+0000","last_peered":"2023-03-29T14:03:34.223432+0000","last_clean":"2023-03-29T14:03:34.223432+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:34.223432+0000","last_undegraded":"2023-03-29T14:03:34.223432+0000","last_fullsized":"2023-03-29T14:03:34.223432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'49","reported_seq":191,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.703390+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:35.703390+0000","last_peered":"2023-03-29T14:03:35.703390+0000","last_clean":"2023-03-29T14:03:35.703390+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:35.703390+0000","last_undegraded":"2023-03-29T14:03:35.703390+0000","last_fullsized":"2023-03-29T14:03:35.703390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378372,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":94,"num_read_kb":53127,"num_write":104,"num_write_kb":30648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:34.211316+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:34.211316+0000","last_peered":"2023-03-29T14:03:34.211316+0000","last_clean":"2023-03-29T14:03:34.211316+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:34.211316+0000","last_undegraded":"2023-03-29T14:03:34.211316+0000","last_fullsized":"2023-03-29T14:03:34.211316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"85'31","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.613274+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:35.613274+0000","last_peered":"2023-03-29T14:03:35.613274+0000","last_clean":"2023-03-29T14:03:35.613274+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:35.613274+0000","last_undegraded":"2023-03-29T14:03:35.613274+0000","last_fullsized":"2023-03-29T14:03:35.613274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16571636,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":33441,"num_write":68,"num_write_kb":18202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.599609+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:35.599609+0000","last_peered":"2023-03-29T14:03:35.599609+0000","last_clean":"2023-03-29T14:03:35.599609+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:35.599609+0000","last_undegraded":"2023-03-29T14:03:35.599609+0000","last_fullsized":"2023-03-29T14:03:35.599609+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.326118+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:35.326118+0000","last_peered":"2023-03-29T14:03:35.326118+0000","last_clean":"2023-03-29T14:03:35.326118+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:35.326118+0000","last_undegraded":"2023-03-29T14:03:35.326118+0000","last_fullsized":"2023-03-29T14:03:35.326118+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'26","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-03-29T14:03:33.244320+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:33.244320+0000","last_peered":"2023-03-29T14:03:33.244320+0000","last_clean":"2023-03-29T14:03:33.244320+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:33.244320+0000","last_undegraded":"2023-03-29T14:03:33.244320+0000","last_fullsized":"2023-03-29T14:03:33.244320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5008486,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:34.220532+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:34.220532+0000","last_peered":"2023-03-29T14:03:34.220532+0000","last_clean":"2023-03-29T14:03:34.220532+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:34.220532+0000","last_undegraded":"2023-03-29T14:03:34.220532+0000","last_fullsized":"2023-03-29T14:03:34.220532+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"90'24","reported_seq":133,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.639183+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:35.639183+0000","last_peered":"2023-03-29T14:03:35.639183+0000","last_clean":"2023-03-29T14:03:35.639183+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:35.639183+0000","last_undegraded":"2023-03-29T14:03:35.639183+0000","last_fullsized":"2023-03-29T14:03:35.639183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12470794,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":31040,"num_write":54,"num_write_kb":13922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-03-29T14:03:33.234776+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:33.234776+0000","last_peered":"2023-03-29T14:03:33.234776+0000","last_clean":"2023-03-29T14:03:33.234776+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:33.234776+0000","last_undegraded":"2023-03-29T14:03:33.234776+0000","last_fullsized":"2023-03-29T14:03:33.234776+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"90'43","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.671390+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:35.671390+0000","last_peered":"2023-03-29T14:03:35.671390+0000","last_clean":"2023-03-29T14:03:35.671390+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:35.671390+0000","last_undegraded":"2023-03-29T14:03:35.671390+0000","last_fullsized":"2023-03-29T14:03:35.671390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11832128,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":45584,"num_write":90,"num_write_kb":27430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:34.218781+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:34.218781+0000","last_peered":"2023-03-29T14:03:34.218781+0000","last_clean":"2023-03-29T14:03:34.218781+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:34.218781+0000","last_undegraded":"2023-03-29T14:03:34.218781+0000","last_fullsized":"2023-03-29T14:03:34.218781+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":296,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.318955+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:35.318955+0000","last_peered":"2023-03-29T14:03:35.318955+0000","last_clean":"2023-03-29T14:03:35.318955+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:35.318955+0000","last_undegraded":"2023-03-29T14:03:35.318955+0000","last_fullsized":"2023-03-29T14:03:35.318955+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":199,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:34.213129+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:34.213129+0000","last_peered":"2023-03-29T14:03:34.213129+0000","last_clean":"2023-03-29T14:03:34.213129+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:34.213129+0000","last_undegraded":"2023-03-29T14:03:34.213129+0000","last_fullsized":"2023-03-29T14:03:34.213129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":51659,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:34.210064+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:34.210064+0000","last_peered":"2023-03-29T14:03:34.210064+0000","last_clean":"2023-03-29T14:03:34.210064+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:34.210064+0000","last_undegraded":"2023-03-29T14:03:34.210064+0000","last_fullsized":"2023-03-29T14:03:34.210064+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"77'96","reported_seq":290,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.351913+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:35.351913+0000","last_peered":"2023-03-29T14:03:35.351913+0000","last_clean":"2023-03-29T14:03:35.351913+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:35.351913+0000","last_undegraded":"2023-03-29T14:03:35.351913+0000","last_fullsized":"2023-03-29T14:03:35.351913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630510,"num_objects":34,"num_object_clones":22,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":129,"num_read_kb":61112,"num_write":196,"num_write_kb":55794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.336876+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:35.336876+0000","last_peered":"2023-03-29T14:03:35.336876+0000","last_clean":"2023-03-29T14:03:35.336876+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:35.336876+0000","last_undegraded":"2023-03-29T14:03:35.336876+0000","last_fullsized":"2023-03-29T14:03:35.336876+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.290303+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:35.290303+0000","last_peered":"2023-03-29T14:03:35.290303+0000","last_clean":"2023-03-29T14:03:35.290303+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:35.290303+0000","last_undegraded":"2023-03-29T14:03:35.290303+0000","last_fullsized":"2023-03-29T14:03:35.290303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"90'47","reported_seq":173,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.761071+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:35.761071+0000","last_peered":"2023-03-29T14:03:35.761071+0000","last_clean":"2023-03-29T14:03:35.761071+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:35.761071+0000","last_undegraded":"2023-03-29T14:03:35.761071+0000","last_fullsized":"2023-03-29T14:03:35.761071+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26790292,"num_objects":20,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":51,"num_read_kb":18422,"num_write":104,"num_write_kb":31140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.310922+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:35.310922+0000","last_peered":"2023-03-29T14:03:35.310922+0000","last_clean":"2023-03-29T14:03:35.310922+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:35.310922+0000","last_undegraded":"2023-03-29T14:03:35.310922+0000","last_fullsized":"2023-03-29T14:03:35.310922+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-03-29T14:03:34.224143+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:34.224143+0000","last_peered":"2023-03-29T14:03:34.224143+0000","last_clean":"2023-03-29T14:03:34.224143+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:34.224143+0000","last_undegraded":"2023-03-29T14:03:34.224143+0000","last_fullsized":"2023-03-29T14:03:34.224143+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"82'41","reported_seq":176,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-03-29T14:03:35.270116+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:35.270116+0000","last_peered":"2023-03-29T14:03:35.270116+0000","last_clean":"2023-03-29T14:03:35.270116+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:35.270116+0000","last_undegraded":"2023-03-29T14:03:35.270116+0000","last_fullsized":"2023-03-29T14:03:35.270116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19563282,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":45125,"num_write":86,"num_write_kb":23154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":208574408,"num_objects":250,"num_object_clones":152,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":983,"num_read_kb":517589,"num_write":1302,"num_write_kb":372986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":619,"ondisk_log_size":619,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":128220,"kb_used_data":121300,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94243620,"statfs":{"total":96636764160,"available":96505466880,"internally_reserved":0,"allocated":124211200,"data_stored":143327552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":84320,"kb_used_data":77592,"kb_used_omap":0,"kb_used_meta":6720,"kb_avail":94287520,"statfs":{"total":96636764160,"available":96550420480,"internally_reserved":0,"allocated":79454208,"data_stored":88889377,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6881280},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":133468,"kb_used_data":125972,"kb_used_omap":0,"kb_used_meta":7488,"kb_avail":94238372,"statfs":{"total":96636764160,"available":96500092928,"internally_reserved":0,"allocated":128995328,"data_stored":149466867,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7667712},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":128840,"kb_used_data":121600,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94243000,"statfs":{"total":96636764160,"available":96504832000,"internally_reserved":0,"allocated":124518400,"data_stored":142915386,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:40.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:40.267 INFO:tasks.rados.rados.0.smithi017.stdout:374: done (4 left) 2023-03-29T14:03:40.268 INFO:tasks.rados.rados.0.smithi017.stdout:375: done (3 left) 2023-03-29T14:03:40.268 INFO:tasks.rados.rados.0.smithi017.stdout:376: done (2 left) 2023-03-29T14:03:40.268 INFO:tasks.rados.rados.0.smithi017.stdout:377: done (1 left) 2023-03-29T14:03:40.268 INFO:tasks.rados.rados.0.smithi017.stdout:378: done (0 left) 2023-03-29T14:03:40.268 INFO:tasks.rados.rados.0.smithi017.stdout:379: delete oid 8 current snap is 42 2023-03-29T14:03:40.300 INFO:tasks.rados.rados.0.smithi017.stdout:379: done (0 left) 2023-03-29T14:03:40.300 INFO:tasks.rados.rados.0.smithi017.stdout:380: read oid 14 snap -1 2023-03-29T14:03:40.300 INFO:tasks.rados.rados.0.smithi017.stdout:380: expect (ObjNum 187 snap 36 seq_num 187) 2023-03-29T14:03:40.300 INFO:tasks.rados.rados.0.smithi017.stdout:381: snap_create 2023-03-29T14:03:40.319 INFO:tasks.rados.rados.0.smithi017.stdout:380: expect (ObjNum 187 snap 36 seq_num 187) 2023-03-29T14:03:40.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:40.355+0000 7f15a9f00700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:40.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:40.360+0000 7f15a9f00700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:41.271 INFO:tasks.rados.rados.0.smithi017.stdout:380: done (1 left) 2023-03-29T14:03:41.271 INFO:tasks.rados.rados.0.smithi017.stdout:381: done (0 left) 2023-03-29T14:03:41.271 INFO:tasks.rados.rados.0.smithi017.stdout:382: snap_remove snap 40 2023-03-29T14:03:42.276 INFO:tasks.rados.rados.0.smithi017.stdout:382: done (0 left) 2023-03-29T14:03:42.276 INFO:tasks.rados.rados.0.smithi017.stdout:383: write (excl) oid 17 current snap is 43 2023-03-29T14:03:42.277 INFO:tasks.rados.rados.0.smithi017.stdout:383: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2023-03-29T14:03:42.282 INFO:tasks.rados.rados.0.smithi017.stdout:383: writing smithi017107884-17 from 578207 to 1128434 tid 1 2023-03-29T14:03:42.291 INFO:tasks.rados.rados.0.smithi017.stdout:383: writing smithi017107884-17 from 1762410 to 2542144 tid 2 2023-03-29T14:03:42.295 INFO:tasks.rados.rados.0.smithi017.stdout:383: writing smithi017107884-17 from 3286845 to 3378207 tid 3 2023-03-29T14:03:42.295 INFO:tasks.rados.rados.0.smithi017.stdout:384: read oid 20 snap -1 2023-03-29T14:03:42.295 INFO:tasks.rados.rados.0.smithi017.stdout:384: expect (ObjNum 153 snap 21 seq_num 153) 2023-03-29T14:03:42.295 INFO:tasks.rados.rados.0.smithi017.stdout:385: delete oid 42 current snap is 43 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:385: done (2 left) 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:386: read oid 12 snap 41 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:386: expect (ObjNum 197 snap 40 seq_num 197) 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:387: read oid 46 snap -1 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:387: expect deleted 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:388: read oid 47 snap -1 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:388: expect (ObjNum 192 snap 38 seq_num 192) 2023-03-29T14:03:42.368 INFO:tasks.rados.rados.0.smithi017.stdout:389: read oid 26 snap -1 2023-03-29T14:03:42.369 INFO:tasks.rados.rados.0.smithi017.stdout:389: expect (ObjNum 180 snap 33 seq_num 180) 2023-03-29T14:03:42.369 INFO:tasks.rados.rados.0.smithi017.stdout:390: read oid 43 snap -1 2023-03-29T14:03:42.369 INFO:tasks.rados.rados.0.smithi017.stdout:390: expect deleted 2023-03-29T14:03:42.369 INFO:tasks.rados.rados.0.smithi017.stdout:391: snap_create 2023-03-29T14:03:42.384 INFO:tasks.rados.rados.0.smithi017.stdout:389: expect (ObjNum 180 snap 33 seq_num 180) 2023-03-29T14:03:42.434 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:42.507 INFO:tasks.rados.rados.0.smithi017.stdout:384: expect (ObjNum 153 snap 21 seq_num 153) 2023-03-29T14:03:42.569 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:42.567+0000 7f4f44096700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:42.572 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:42.571+0000 7f4f44096700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:42.681 INFO:tasks.rados.rados.0.smithi017.stdout:383: finishing write tid 1 to smithi017107884-17 2023-03-29T14:03:42.682 INFO:tasks.rados.rados.0.smithi017.stdout:386: expect (ObjNum 197 snap 40 seq_num 197) 2023-03-29T14:03:42.847 INFO:tasks.rados.rados.0.smithi017.stdout:383: finishing write tid 2 to smithi017107884-17 2023-03-29T14:03:42.847 INFO:tasks.rados.rados.0.smithi017.stdout:383: finishing write tid 3 to smithi017107884-17 2023-03-29T14:03:42.847 INFO:tasks.rados.rados.0.smithi017.stdout:383: finishing write tid 4 to smithi017107884-17 2023-03-29T14:03:42.847 INFO:tasks.rados.rados.0.smithi017.stdout:388: expect (ObjNum 192 snap 38 seq_num 192) 2023-03-29T14:03:43.009 INFO:tasks.rados.rados.0.smithi017.stdout:383: finishing write tid 5 to smithi017107884-17 2023-03-29T14:03:43.010 INFO:tasks.rados.rados.0.smithi017.stdout:383: oid 17 updating version 0 to 69 2023-03-29T14:03:43.010 INFO:tasks.rados.rados.0.smithi017.stdout:383: oid 17 version 69 is already newer than 65 2023-03-29T14:03:43.010 INFO:tasks.rados.rados.0.smithi017.stdout:383: oid 17 version 69 is already newer than 69 2023-03-29T14:03:43.010 INFO:tasks.rados.rados.0.smithi017.stdout:383: oid 17 version 69 is already newer than 69 2023-03-29T14:03:43.010 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 69 (ObjNum 201 snap 43 seq_num 201) dirty exists 2023-03-29T14:03:43.010 INFO:tasks.rados.rados.0.smithi017.stdout:383: left oid 17 (ObjNum 201 snap 43 seq_num 201) 2023-03-29T14:03:43.185 INFO:tasks.rados.rados.0.smithi017.stdout:383: done (7 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:384: done (6 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:386: done (5 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:387: done (4 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:388: done (3 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:389: done (2 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:390: done (1 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:391: done (0 left) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:392: read oid 26 snap 37 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:392: expect (ObjNum 180 snap 33 seq_num 180) 2023-03-29T14:03:43.186 INFO:tasks.rados.rados.0.smithi017.stdout:393: delete oid 29 current snap is 44 2023-03-29T14:03:43.223 INFO:tasks.rados.rados.0.smithi017.stdout:392: expect (ObjNum 180 snap 33 seq_num 180) 2023-03-29T14:03:43.344 INFO:tasks.rados.rados.0.smithi017.stdout:392: done (1 left) 2023-03-29T14:03:43.344 INFO:tasks.rados.rados.0.smithi017.stdout:393: done (0 left) 2023-03-29T14:03:43.345 INFO:tasks.rados.rados.0.smithi017.stdout:394: write oid 4 current snap is 44 2023-03-29T14:03:43.345 INFO:tasks.rados.rados.0.smithi017.stdout:394: seq_num 204 ranges {628628=775974,1937856=527061,3252761=175867} 2023-03-29T14:03:43.352 INFO:tasks.rados.rados.0.smithi017.stdout:394: writing smithi017107884-4 from 628628 to 1404602 tid 1 2023-03-29T14:03:43.358 INFO:tasks.rados.rados.0.smithi017.stdout:394: writing smithi017107884-4 from 1937856 to 2464917 tid 2 2023-03-29T14:03:43.363 INFO:tasks.rados.rados.0.smithi017.stdout:394: writing smithi017107884-4 from 3252761 to 3428628 tid 3 2023-03-29T14:03:43.364 INFO:tasks.rados.rados.0.smithi017.stdout:395: read oid 29 snap -1 2023-03-29T14:03:43.364 INFO:tasks.rados.rados.0.smithi017.stdout:395: expect deleted 2023-03-29T14:03:43.364 INFO:tasks.rados.rados.0.smithi017.stdout:396: read oid 44 snap -1 2023-03-29T14:03:43.364 INFO:tasks.rados.rados.0.smithi017.stdout:396: expect (ObjNum 105 snap 9 seq_num 105) 2023-03-29T14:03:43.364 INFO:tasks.rados.rados.0.smithi017.stdout:397: write (excl) oid 23 current snap is 44 2023-03-29T14:03:43.364 INFO:tasks.rados.rados.0.smithi017.stdout:397: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2023-03-29T14:03:43.370 INFO:tasks.rados.rados.0.smithi017.stdout:397: writing smithi017107884-23 from 645435 to 1096658 tid 1 2023-03-29T14:03:43.378 INFO:tasks.rados.rados.0.smithi017.stdout:397: writing smithi017107884-23 from 1879985 to 2667057 tid 2 2023-03-29T14:03:43.383 INFO:tasks.rados.rados.0.smithi017.stdout:397: writing smithi017107884-23 from 3280184 to 3445435 tid 3 2023-03-29T14:03:43.383 INFO:tasks.rados.rados.0.smithi017.stdout:398: delete oid 31 current snap is 44 2023-03-29T14:03:43.451 INFO:tasks.rados.rados.0.smithi017.stdout:394: finishing write tid 1 to smithi017107884-4 2023-03-29T14:03:43.451 INFO:tasks.rados.rados.0.smithi017.stdout:394: finishing write tid 2 to smithi017107884-4 2023-03-29T14:03:43.493 INFO:tasks.rados.rados.0.smithi017.stdout:396: expect (ObjNum 105 snap 9 seq_num 105) 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:397: finishing write tid 1 to smithi017107884-23 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: finishing write tid 3 to smithi017107884-4 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: finishing write tid 4 to smithi017107884-4 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: finishing write tid 5 to smithi017107884-4 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: oid 4 updating version 0 to 74 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: oid 4 version 74 is already newer than 74 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: oid 4 version 74 is already newer than 74 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: oid 4 version 74 is already newer than 74 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 74 (ObjNum 204 snap 44 seq_num 204) dirty exists 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:394: left oid 4 (ObjNum 204 snap 44 seq_num 204) 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:397: finishing write tid 2 to smithi017107884-23 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:397: finishing write tid 3 to smithi017107884-23 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:397: finishing write tid 4 to smithi017107884-23 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:397: finishing write tid 5 to smithi017107884-23 2023-03-29T14:03:43.723 INFO:tasks.rados.rados.0.smithi017.stdout:397: oid 23 updating version 0 to 103 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:397: oid 23 updating version 103 to 106 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:397: oid 23 version 106 is already newer than 106 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:397: oid 23 version 106 is already newer than 106 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 106 (ObjNum 205 snap 44 seq_num 205) dirty exists 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:397: left oid 23 (ObjNum 205 snap 44 seq_num 205) 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:394: done (4 left) 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:395: done (3 left) 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:396: done (2 left) 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:397: done (1 left) 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:398: done (0 left) 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:399: write oid 31 current snap is 44 2023-03-29T14:03:43.724 INFO:tasks.rados.rados.0.smithi017.stdout:399: seq_num 207 ranges {679049=718074,2113302=674388,3335030=144019} 2023-03-29T14:03:43.730 INFO:tasks.rados.rados.0.smithi017.stdout:399: writing smithi017107884-31 from 679049 to 1397123 tid 1 2023-03-29T14:03:43.738 INFO:tasks.rados.rados.0.smithi017.stdout:399: writing smithi017107884-31 from 2113302 to 2787690 tid 2 2023-03-29T14:03:43.742 INFO:tasks.rados.rados.0.smithi017.stdout:399: writing smithi017107884-31 from 3335030 to 3479049 tid 3 2023-03-29T14:03:43.742 INFO:tasks.rados.rados.0.smithi017.stdout:400: write oid 24 current snap is 44 2023-03-29T14:03:43.742 INFO:tasks.rados.rados.0.smithi017.stdout:400: seq_num 208 ranges {695856=793323,2171784=534399,3362453=133403} 2023-03-29T14:03:43.750 INFO:tasks.rados.rados.0.smithi017.stdout:400: writing smithi017107884-24 from 695856 to 1489179 tid 1 2023-03-29T14:03:43.757 INFO:tasks.rados.rados.0.smithi017.stdout:400: writing smithi017107884-24 from 2171784 to 2706183 tid 2 2023-03-29T14:03:43.761 INFO:tasks.rados.rados.0.smithi017.stdout:400: writing smithi017107884-24 from 3362453 to 3495856 tid 3 2023-03-29T14:03:43.761 INFO:tasks.rados.rados.0.smithi017.stdout:401: read oid 15 snap -1 2023-03-29T14:03:43.762 INFO:tasks.rados.rados.0.smithi017.stdout:401: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:03:43.762 INFO:tasks.rados.rados.0.smithi017.stdout:402: snap_create 2023-03-29T14:03:43.771 INFO:tasks.rados.rados.0.smithi017.stdout:399: finishing write tid 1 to smithi017107884-31 2023-03-29T14:03:43.785 INFO:tasks.rados.rados.0.smithi017.stdout:399: finishing write tid 2 to smithi017107884-31 2023-03-29T14:03:43.798 INFO:tasks.rados.rados.0.smithi017.stdout:400: finishing write tid 1 to smithi017107884-24 2023-03-29T14:03:43.812 INFO:tasks.rados.rados.0.smithi017.stdout:400: finishing write tid 2 to smithi017107884-24 2023-03-29T14:03:43.826 INFO:tasks.rados.rados.0.smithi017.stdout:399: finishing write tid 3 to smithi017107884-31 2023-03-29T14:03:43.827 INFO:tasks.rados.rados.0.smithi017.stdout:399: finishing write tid 4 to smithi017107884-31 2023-03-29T14:03:43.827 INFO:tasks.rados.rados.0.smithi017.stdout:399: finishing write tid 5 to smithi017107884-31 2023-03-29T14:03:43.828 INFO:tasks.rados.rados.0.smithi017.stdout:399: oid 31 updating version 0 to 37 2023-03-29T14:03:43.828 INFO:tasks.rados.rados.0.smithi017.stdout:399: oid 31 version 37 is already newer than 37 2023-03-29T14:03:43.828 INFO:tasks.rados.rados.0.smithi017.stdout:399: oid 31 version 37 is already newer than 37 2023-03-29T14:03:43.828 INFO:tasks.rados.rados.0.smithi017.stdout:399: oid 31 version 37 is already newer than 35 2023-03-29T14:03:43.828 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 37 (ObjNum 207 snap 44 seq_num 207) dirty exists 2023-03-29T14:03:43.828 INFO:tasks.rados.rados.0.smithi017.stdout:399: left oid 31 (ObjNum 207 snap 44 seq_num 207) 2023-03-29T14:03:43.831 INFO:tasks.rados.rados.0.smithi017.stdout:401: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:03:43.959 INFO:tasks.rados.rados.0.smithi017.stdout:400: finishing write tid 3 to smithi017107884-24 2023-03-29T14:03:43.959 INFO:tasks.rados.rados.0.smithi017.stdout:400: finishing write tid 4 to smithi017107884-24 2023-03-29T14:03:43.959 INFO:tasks.rados.rados.0.smithi017.stdout:400: finishing write tid 5 to smithi017107884-24 2023-03-29T14:03:43.959 INFO:tasks.rados.rados.0.smithi017.stdout:400: oid 24 updating version 0 to 31 2023-03-29T14:03:43.959 INFO:tasks.rados.rados.0.smithi017.stdout:400: oid 24 version 31 is already newer than 31 2023-03-29T14:03:43.960 INFO:tasks.rados.rados.0.smithi017.stdout:400: oid 24 version 31 is already newer than 30 2023-03-29T14:03:43.960 INFO:tasks.rados.rados.0.smithi017.stdout:400: oid 24 version 31 is already newer than 31 2023-03-29T14:03:43.960 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 31 (ObjNum 208 snap 44 seq_num 208) dirty exists 2023-03-29T14:03:43.960 INFO:tasks.rados.rados.0.smithi017.stdout:400: left oid 24 (ObjNum 208 snap 44 seq_num 208) 2023-03-29T14:03:44.279 INFO:tasks.rados.rados.0.smithi017.stdout:399: done (3 left) 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:400: done (2 left) 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:401: done (1 left) 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:402: done (0 left) 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:403: read oid 17 snap -1 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:403: expect (ObjNum 201 snap 43 seq_num 201) 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:404: read oid 35 snap 37 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:404: expect deleted 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:405: read oid 8 snap -1 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:405: expect deleted 2023-03-29T14:03:44.280 INFO:tasks.rados.rados.0.smithi017.stdout:406: snap_remove snap 44 2023-03-29T14:03:44.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:44.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:44.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:44.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:44.776 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:44.774+0000 7f66c18c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:44.780 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:44.779+0000 7f66c18c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:44.830+0000 7fbe67530700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:44.837+0000 7fbe67530700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:45.047 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:45.048 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:45.060 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2023-03-29T14:03:44.278802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222226070,"num_objects":264,"num_object_clones":162,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1169,"num_read_kb":542493,"num_write":1646,"num_write_kb":399016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":517708,"kb_used_data":484716,"kb_used_omap":0,"kb_used_meta":32960,"kb_avail":376969652,"statfs":{"total":386547056640,"available":386016923648,"internally_reserved":0,"allocated":496349184,"data_stored":571205858,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":33751040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12470926,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-4,"num_read":44,"num_read_kb":24784,"num_write":70,"num_write_kb":22242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012169"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":129,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:38.590715+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:38.590715+0000","last_peered":"2023-03-29T14:03:38.590715+0000","last_clean":"2023-03-29T14:03:38.590715+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:38.590715+0000","last_undegraded":"2023-03-29T14:03:38.590715+0000","last_fullsized":"2023-03-29T14:03:38.590715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":39299,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":92,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.450671+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:39.450671+0000","last_peered":"2023-03-29T14:03:39.450671+0000","last_clean":"2023-03-29T14:03:39.450671+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:39.450671+0000","last_undegraded":"2023-03-29T14:03:39.450671+0000","last_fullsized":"2023-03-29T14:03:39.450671+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10764,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"93'64","reported_seq":213,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.306593+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:40.306593+0000","last_peered":"2023-03-29T14:03:40.306593+0000","last_clean":"2023-03-29T14:03:40.306593+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:40.306593+0000","last_undegraded":"2023-03-29T14:03:40.306593+0000","last_fullsized":"2023-03-29T14:03:40.306593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983556,"num_objects":20,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":69,"num_read_kb":31556,"num_write":134,"num_write_kb":39254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":194,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.295828+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:40.295828+0000","last_peered":"2023-03-29T14:03:40.295828+0000","last_clean":"2023-03-29T14:03:40.295828+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:40.295828+0000","last_undegraded":"2023-03-29T14:03:40.295828+0000","last_fullsized":"2023-03-29T14:03:40.295828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"93'55","reported_seq":204,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.353532+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:39.353532+0000","last_peered":"2023-03-29T14:03:39.353532+0000","last_clean":"2023-03-29T14:03:39.353532+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:39.353532+0000","last_undegraded":"2023-03-29T14:03:39.353532+0000","last_fullsized":"2023-03-29T14:03:39.353532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773386,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":53128,"num_write":114,"num_write_kb":33984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":72,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.294526+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:40.294526+0000","last_peered":"2023-03-29T14:03:40.294526+0000","last_clean":"2023-03-29T14:03:40.294526+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:40.294526+0000","last_undegraded":"2023-03-29T14:03:40.294526+0000","last_fullsized":"2023-03-29T14:03:40.294526+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"85'31","reported_seq":155,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.318201+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:40.318201+0000","last_peered":"2023-03-29T14:03:40.318201+0000","last_clean":"2023-03-29T14:03:40.318201+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:40.318201+0000","last_undegraded":"2023-03-29T14:03:40.318201+0000","last_fullsized":"2023-03-29T14:03:40.318201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16571636,"num_objects":14,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":38106,"num_write":68,"num_write_kb":18202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":154,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.297293+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:40.297293+0000","last_peered":"2023-03-29T14:03:40.297293+0000","last_clean":"2023-03-29T14:03:40.297293+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:40.297293+0000","last_undegraded":"2023-03-29T14:03:40.297293+0000","last_fullsized":"2023-03-29T14:03:40.297293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.364312+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:39.364312+0000","last_peered":"2023-03-29T14:03:39.364312+0000","last_clean":"2023-03-29T14:03:39.364312+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:39.364312+0000","last_undegraded":"2023-03-29T14:03:39.364312+0000","last_fullsized":"2023-03-29T14:03:39.364312+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'26","reported_seq":113,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:38.322151+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:38.322151+0000","last_peered":"2023-03-29T14:03:38.322151+0000","last_clean":"2023-03-29T14:03:38.322151+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:38.322151+0000","last_undegraded":"2023-03-29T14:03:38.322151+0000","last_fullsized":"2023-03-29T14:03:38.322151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5008486,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":13759,"num_write":48,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":75,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.291907+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:40.291907+0000","last_peered":"2023-03-29T14:03:40.291907+0000","last_clean":"2023-03-29T14:03:40.291907+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:40.291907+0000","last_undegraded":"2023-03-29T14:03:40.291907+0000","last_fullsized":"2023-03-29T14:03:40.291907+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"90'24","reported_seq":138,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.315250+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:39.315250+0000","last_peered":"2023-03-29T14:03:39.315250+0000","last_clean":"2023-03-29T14:03:39.315250+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:39.315250+0000","last_undegraded":"2023-03-29T14:03:39.315250+0000","last_fullsized":"2023-03-29T14:03:39.315250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12470794,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":31040,"num_write":54,"num_write_kb":13922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:38.324584+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:38.324584+0000","last_peered":"2023-03-29T14:03:38.324584+0000","last_clean":"2023-03-29T14:03:38.324584+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:38.324584+0000","last_undegraded":"2023-03-29T14:03:38.324584+0000","last_fullsized":"2023-03-29T14:03:38.324584+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"90'43","reported_seq":180,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.350561+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:39.350561+0000","last_peered":"2023-03-29T14:03:39.350561+0000","last_clean":"2023-03-29T14:03:39.350561+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:39.350561+0000","last_undegraded":"2023-03-29T14:03:39.350561+0000","last_fullsized":"2023-03-29T14:03:39.350561+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11832128,"num_objects":18,"num_object_clones":12,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":80,"num_read_kb":45584,"num_write":90,"num_write_kb":27430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":75,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.296478+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:40.296478+0000","last_peered":"2023-03-29T14:03:40.296478+0000","last_clean":"2023-03-29T14:03:40.296478+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:40.296478+0000","last_undegraded":"2023-03-29T14:03:40.296478+0000","last_fullsized":"2023-03-29T14:03:40.296478+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":300,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.334300+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:39.334300+0000","last_peered":"2023-03-29T14:03:39.334300+0000","last_clean":"2023-03-29T14:03:39.334300+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:39.334300+0000","last_undegraded":"2023-03-29T14:03:39.334300+0000","last_fullsized":"2023-03-29T14:03:39.334300+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":203,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.287021+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:40.287021+0000","last_peered":"2023-03-29T14:03:40.287021+0000","last_clean":"2023-03-29T14:03:40.287021+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:40.287021+0000","last_undegraded":"2023-03-29T14:03:40.287021+0000","last_fullsized":"2023-03-29T14:03:40.287021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":51659,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.301865+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:40.301865+0000","last_peered":"2023-03-29T14:03:40.301865+0000","last_clean":"2023-03-29T14:03:40.301865+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:40.301865+0000","last_undegraded":"2023-03-29T14:03:40.301865+0000","last_fullsized":"2023-03-29T14:03:40.301865+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"93'102","reported_seq":307,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.446730+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:39.446730+0000","last_peered":"2023-03-29T14:03:39.446730+0000","last_clean":"2023-03-29T14:03:39.446730+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:39.446730+0000","last_undegraded":"2023-03-29T14:03:39.446730+0000","last_fullsized":"2023-03-29T14:03:39.446730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22218854,"num_objects":34,"num_object_clones":22,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":137,"num_read_kb":69633,"num_write":206,"num_write_kb":58754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.300082+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:39.300082+0000","last_peered":"2023-03-29T14:03:39.300082+0000","last_clean":"2023-03-29T14:03:39.300082+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:39.300082+0000","last_undegraded":"2023-03-29T14:03:39.300082+0000","last_fullsized":"2023-03-29T14:03:39.300082+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":155,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.312946+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:39.312946+0000","last_peered":"2023-03-29T14:03:39.312946+0000","last_clean":"2023-03-29T14:03:39.312946+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:39.312946+0000","last_undegraded":"2023-03-29T14:03:39.312946+0000","last_fullsized":"2023-03-29T14:03:39.312946+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":57,"num_read_kb":34137,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"93'52","reported_seq":186,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.362467+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:39.362467+0000","last_peered":"2023-03-29T14:03:39.362467+0000","last_clean":"2023-03-29T14:03:39.362467+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:39.362467+0000","last_undegraded":"2023-03-29T14:03:39.362467+0000","last_fullsized":"2023-03-29T14:03:39.362467+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193660,"num_objects":22,"num_object_clones":12,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":54,"num_read_kb":18423,"num_write":114,"num_write_kb":34182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:39.340151+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:39.340151+0000","last_peered":"2023-03-29T14:03:39.340151+0000","last_clean":"2023-03-29T14:03:39.340151+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:39.340151+0000","last_undegraded":"2023-03-29T14:03:39.340151+0000","last_fullsized":"2023-03-29T14:03:39.340151+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.292605+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:40.292605+0000","last_peered":"2023-03-29T14:03:40.292605+0000","last_clean":"2023-03-29T14:03:40.292605+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:40.292605+0000","last_undegraded":"2023-03-29T14:03:40.292605+0000","last_fullsized":"2023-03-29T14:03:40.292605+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'51","reported_seq":200,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-03-29T14:03:40.296852+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:40.296852+0000","last_peered":"2023-03-29T14:03:40.296852+0000","last_clean":"2023-03-29T14:03:40.296852+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:40.296852+0000","last_undegraded":"2023-03-29T14:03:40.296852+0000","last_fullsized":"2023-03-29T14:03:40.296852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20571702,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":45127,"num_write":106,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":221045334,"num_objects":260,"num_object_clones":162,"num_object_copies":780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1027,"num_read_kb":542373,"num_write":1372,"num_write_kb":395228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":655,"ondisk_log_size":655,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":135976,"kb_used_data":128928,"kb_used_omap":0,"kb_used_meta":7040,"kb_avail":94235864,"statfs":{"total":96636764160,"available":96497524736,"internally_reserved":0,"allocated":132022272,"data_stored":152740315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7208960},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":90852,"kb_used_data":84060,"kb_used_omap":0,"kb_used_meta":6784,"kb_avail":94280988,"statfs":{"total":96636764160,"available":96543731712,"internally_reserved":0,"allocated":86077440,"data_stored":95899624,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":6946816},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":153552,"kb_used_data":141768,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94218288,"statfs":{"total":96636764160,"available":96479526912,"internally_reserved":0,"allocated":145170432,"data_stored":169039483,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137328,"kb_used_data":129960,"kb_used_omap":0,"kb_used_meta":7360,"kb_avail":94234512,"statfs":{"total":96636764160,"available":96496140288,"internally_reserved":0,"allocated":133079040,"data_stored":153526436,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7536640},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:45.281 INFO:tasks.rados.rados.0.smithi017.stdout:406: done (3 left) 2023-03-29T14:03:45.281 INFO:tasks.rados.rados.0.smithi017.stdout:407: delete oid 4 current snap is 45 2023-03-29T14:03:45.281 INFO:tasks.rados.rados.0.smithi017.stdout:403: expect (ObjNum 201 snap 43 seq_num 201) 2023-03-29T14:03:45.457 INFO:tasks.rados.rados.0.smithi017.stdout:403: done (3 left) 2023-03-29T14:03:45.457 INFO:tasks.rados.rados.0.smithi017.stdout:404: done (2 left) 2023-03-29T14:03:45.457 INFO:tasks.rados.rados.0.smithi017.stdout:407: done (1 left) 2023-03-29T14:03:45.457 INFO:tasks.rados.rados.0.smithi017.stdout:408: write oid 36 current snap is 45 2023-03-29T14:03:45.457 INFO:tasks.rados.rados.0.smithi017.stdout:408: seq_num 210 ranges {729470=543821,1772395=538068,2900946=403005} 2023-03-29T14:03:45.464 INFO:tasks.rados.rados.0.smithi017.stdout:408: writing smithi017107884-36 from 729470 to 1273291 tid 1 2023-03-29T14:03:45.469 INFO:tasks.rados.rados.0.smithi017.stdout:408: writing smithi017107884-36 from 1772395 to 2310463 tid 2 2023-03-29T14:03:45.475 INFO:tasks.rados.rados.0.smithi017.stdout:408: writing smithi017107884-36 from 2900946 to 3303951 tid 3 2023-03-29T14:03:45.475 INFO:tasks.rados.rados.0.smithi017.stdout:405: done (1 left) 2023-03-29T14:03:45.475 INFO:tasks.rados.rados.0.smithi017.stdout:409: write (excl) oid 13 current snap is 45 2023-03-29T14:03:45.475 INFO:tasks.rados.rados.0.smithi017.stdout:409: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2023-03-29T14:03:45.482 INFO:tasks.rados.rados.0.smithi017.stdout:409: writing smithi017107884-13 from 746277 to 1365347 tid 1 2023-03-29T14:03:45.489 INFO:tasks.rados.rados.0.smithi017.stdout:409: writing smithi017107884-13 from 1830877 to 2512603 tid 2 2023-03-29T14:03:45.494 INFO:tasks.rados.rados.0.smithi017.stdout:409: writing smithi017107884-13 from 2928369 to 3546277 tid 3 2023-03-29T14:03:45.495 INFO:tasks.rados.rados.0.smithi017.stdout:410: read oid 26 snap -1 2023-03-29T14:03:45.495 INFO:tasks.rados.rados.0.smithi017.stdout:410: expect (ObjNum 180 snap 33 seq_num 180) 2023-03-29T14:03:45.495 INFO:tasks.rados.rados.0.smithi017.stdout:411: snap_remove snap 37 2023-03-29T14:03:46.289 INFO:tasks.rados.rados.0.smithi017.stdout:411: done (3 left) 2023-03-29T14:03:46.289 INFO:tasks.rados.rados.0.smithi017.stdout:412: write (excl) oid 30 current snap is 45 2023-03-29T14:03:46.289 INFO:tasks.rados.rados.0.smithi017.stdout:412: seq_num 212 ranges {763084=694319,1889359=541737,2955792=541902} 2023-03-29T14:03:46.296 INFO:tasks.rados.rados.0.smithi017.stdout:412: writing smithi017107884-30 from 763084 to 1457403 tid 1 2023-03-29T14:03:46.302 INFO:tasks.rados.rados.0.smithi017.stdout:412: writing smithi017107884-30 from 1889359 to 2431096 tid 2 2023-03-29T14:03:46.308 INFO:tasks.rados.rados.0.smithi017.stdout:412: writing smithi017107884-30 from 2955792 to 3497694 tid 3 2023-03-29T14:03:46.308 INFO:tasks.rados.rados.0.smithi017.stdout:413: snap_create 2023-03-29T14:03:46.308 INFO:tasks.rados.rados.0.smithi017.stdout:410: expect (ObjNum 180 snap 33 seq_num 180) 2023-03-29T14:03:46.427 INFO:tasks.rados.rados.0.smithi017.stdout:408: finishing write tid 1 to smithi017107884-36 2023-03-29T14:03:46.427 INFO:tasks.rados.rados.0.smithi017.stdout:408: finishing write tid 2 to smithi017107884-36 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: finishing write tid 3 to smithi017107884-36 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: finishing write tid 4 to smithi017107884-36 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: finishing write tid 5 to smithi017107884-36 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: oid 36 updating version 0 to 46 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: oid 36 updating version 46 to 48 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: oid 36 version 48 is already newer than 48 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: oid 36 version 48 is already newer than 48 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 48 (ObjNum 210 snap 45 seq_num 210) dirty exists 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:408: left oid 36 (ObjNum 210 snap 45 seq_num 210) 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: finishing write tid 1 to smithi017107884-13 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: finishing write tid 2 to smithi017107884-13 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: finishing write tid 3 to smithi017107884-13 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: finishing write tid 4 to smithi017107884-13 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: finishing write tid 5 to smithi017107884-13 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: oid 13 updating version 0 to 53 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: oid 13 updating version 53 to 57 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: oid 13 version 57 is already newer than 57 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:409: oid 13 version 57 is already newer than 57 2023-03-29T14:03:46.428 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 57 (ObjNum 211 snap 45 seq_num 211) dirty exists 2023-03-29T14:03:46.429 INFO:tasks.rados.rados.0.smithi017.stdout:409: left oid 13 (ObjNum 211 snap 45 seq_num 211) 2023-03-29T14:03:46.429 INFO:tasks.rados.rados.0.smithi017.stdout:412: finishing write tid 1 to smithi017107884-30 2023-03-29T14:03:46.485 INFO:tasks.rados.rados.0.smithi017.stdout:412: finishing write tid 2 to smithi017107884-30 2023-03-29T14:03:46.486 INFO:tasks.rados.rados.0.smithi017.stdout:412: finishing write tid 3 to smithi017107884-30 2023-03-29T14:03:46.486 INFO:tasks.rados.rados.0.smithi017.stdout:412: finishing write tid 4 to smithi017107884-30 2023-03-29T14:03:46.487 INFO:tasks.rados.rados.0.smithi017.stdout:412: finishing write tid 5 to smithi017107884-30 2023-03-29T14:03:46.487 INFO:tasks.rados.rados.0.smithi017.stdout:412: oid 30 updating version 0 to 31 2023-03-29T14:03:46.487 INFO:tasks.rados.rados.0.smithi017.stdout:412: oid 30 version 31 is already newer than 31 2023-03-29T14:03:46.487 INFO:tasks.rados.rados.0.smithi017.stdout:412: oid 30 version 31 is already newer than 27 2023-03-29T14:03:46.487 INFO:tasks.rados.rados.0.smithi017.stdout:412: oid 30 version 31 is already newer than 31 2023-03-29T14:03:46.488 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 31 (ObjNum 212 snap 45 seq_num 212) dirty exists 2023-03-29T14:03:46.488 INFO:tasks.rados.rados.0.smithi017.stdout:412: left oid 30 (ObjNum 212 snap 45 seq_num 212) 2023-03-29T14:03:46.867 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:47.000 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:46.998+0000 7f6390187700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:47.003 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:47.002+0000 7f6390187700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:47.292 INFO:tasks.rados.rados.0.smithi017.stdout:408: done (4 left) 2023-03-29T14:03:47.292 INFO:tasks.rados.rados.0.smithi017.stdout:409: done (3 left) 2023-03-29T14:03:47.292 INFO:tasks.rados.rados.0.smithi017.stdout:410: done (2 left) 2023-03-29T14:03:47.293 INFO:tasks.rados.rados.0.smithi017.stdout:412: done (1 left) 2023-03-29T14:03:47.293 INFO:tasks.rados.rados.0.smithi017.stdout:413: done (0 left) 2023-03-29T14:03:47.293 INFO:tasks.rados.rados.0.smithi017.stdout:414: snap_remove snap 43 2023-03-29T14:03:48.296 INFO:tasks.rados.rados.0.smithi017.stdout:414: done (0 left) 2023-03-29T14:03:48.296 INFO:tasks.rados.rados.0.smithi017.stdout:415: read oid 22 snap -1 2023-03-29T14:03:48.296 INFO:tasks.rados.rados.0.smithi017.stdout:415: expect (ObjNum 194 snap 40 seq_num 194) 2023-03-29T14:03:48.296 INFO:tasks.rados.rados.0.smithi017.stdout:416: snap_remove snap 38 2023-03-29T14:03:49.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:49.214+0000 7f7c9fcfb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:49.219 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:49.218+0000 7f7c9fcfb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:49.296 INFO:tasks.rados.rados.0.smithi017.stdout:416: done (1 left) 2023-03-29T14:03:49.297 INFO:tasks.rados.rados.0.smithi017.stdout:417: write (excl) oid 14 current snap is 46 2023-03-29T14:03:49.297 INFO:tasks.rados.rados.0.smithi017.stdout:417: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2023-03-29T14:03:49.304 INFO:tasks.rados.rados.0.smithi017.stdout:417: writing smithi017107884-14 from 779891 to 1265812 tid 1 2023-03-29T14:03:49.312 INFO:tasks.rados.rados.0.smithi017.stdout:417: writing smithi017107884-14 from 1947841 to 2633236 tid 2 2023-03-29T14:03:49.316 INFO:tasks.rados.rados.0.smithi017.stdout:417: writing smithi017107884-14 from 3383215 to 3579891 tid 3 2023-03-29T14:03:49.317 INFO:tasks.rados.rados.0.smithi017.stdout:418: write oid 11 current snap is 46 2023-03-29T14:03:49.317 INFO:tasks.rados.rados.0.smithi017.stdout:418: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2023-03-29T14:03:49.324 INFO:tasks.rados.rados.0.smithi017.stdout:418: writing smithi017107884-11 from 796698 to 1357868 tid 1 2023-03-29T14:03:49.330 INFO:tasks.rados.rados.0.smithi017.stdout:418: writing smithi017107884-11 from 2006323 to 2551729 tid 2 2023-03-29T14:03:49.336 INFO:tasks.rados.rados.0.smithi017.stdout:418: writing smithi017107884-11 from 3126991 to 3596698 tid 3 2023-03-29T14:03:49.337 INFO:tasks.rados.rados.0.smithi017.stdout:419: read oid 30 snap -1 2023-03-29T14:03:49.337 INFO:tasks.rados.rados.0.smithi017.stdout:419: expect (ObjNum 212 snap 45 seq_num 212) 2023-03-29T14:03:49.337 INFO:tasks.rados.rados.0.smithi017.stdout:420: snap_create 2023-03-29T14:03:49.337 INFO:tasks.rados.rados.0.smithi017.stdout:415: expect (ObjNum 194 snap 40 seq_num 194) 2023-03-29T14:03:49.495 INFO:tasks.rados.rados.0.smithi017.stdout:417: finishing write tid 1 to smithi017107884-14 2023-03-29T14:03:49.495 INFO:tasks.rados.rados.0.smithi017.stdout:419: expect (ObjNum 212 snap 45 seq_num 212) 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: finishing write tid 1 to smithi017107884-11 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: finishing write tid 2 to smithi017107884-11 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: finishing write tid 3 to smithi017107884-11 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: finishing write tid 4 to smithi017107884-11 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: finishing write tid 5 to smithi017107884-11 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: oid 11 updating version 0 to 110 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: oid 11 version 110 is already newer than 110 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: oid 11 version 110 is already newer than 110 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: oid 11 version 110 is already newer than 110 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 11 v 110 (ObjNum 214 snap 46 seq_num 214) dirty exists 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:418: left oid 11 (ObjNum 214 snap 46 seq_num 214) 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:417: finishing write tid 2 to smithi017107884-14 2023-03-29T14:03:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:417: finishing write tid 3 to smithi017107884-14 2023-03-29T14:03:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:417: finishing write tid 4 to smithi017107884-14 2023-03-29T14:03:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:417: finishing write tid 5 to smithi017107884-14 2023-03-29T14:03:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:417: oid 14 updating version 0 to 42 2023-03-29T14:03:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:417: oid 14 version 42 is already newer than 38 2023-03-29T14:03:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:417: oid 14 version 42 is already newer than 42 2023-03-29T14:03:49.691 INFO:tasks.rados.rados.0.smithi017.stdout:417: oid 14 version 42 is already newer than 42 2023-03-29T14:03:49.691 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 42 (ObjNum 213 snap 46 seq_num 213) dirty exists 2023-03-29T14:03:49.691 INFO:tasks.rados.rados.0.smithi017.stdout:417: left oid 14 (ObjNum 213 snap 46 seq_num 213) 2023-03-29T14:03:50.065 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:50.065 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:50.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:50.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:50.195+0000 7f345da66700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:50.202 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:50.202+0000 7f345da66700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:50.302 INFO:tasks.rados.rados.0.smithi017.stdout:415: done (4 left) 2023-03-29T14:03:50.303 INFO:tasks.rados.rados.0.smithi017.stdout:417: done (3 left) 2023-03-29T14:03:50.303 INFO:tasks.rados.rados.0.smithi017.stdout:418: done (2 left) 2023-03-29T14:03:50.303 INFO:tasks.rados.rados.0.smithi017.stdout:419: done (1 left) 2023-03-29T14:03:50.303 INFO:tasks.rados.rados.0.smithi017.stdout:420: done (0 left) 2023-03-29T14:03:50.303 INFO:tasks.rados.rados.0.smithi017.stdout:421: write (excl) oid 13 current snap is 47 2023-03-29T14:03:50.303 INFO:tasks.rados.rados.0.smithi017.stdout:421: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2023-03-29T14:03:50.308 INFO:tasks.rados.rados.0.smithi017.stdout:421: writing smithi017107884-13 from 413505 to 1049924 tid 1 2023-03-29T14:03:50.315 INFO:tasks.rados.rados.0.smithi017.stdout:421: writing smithi017107884-13 from 1664805 to 2353869 tid 2 2023-03-29T14:03:50.322 INFO:tasks.rados.rados.0.smithi017.stdout:421: writing smithi017107884-13 from 3038061 to 3530132 tid 3 2023-03-29T14:03:50.322 INFO:tasks.rados.rados.0.smithi017.stdout:422: write oid 48 current snap is 47 2023-03-29T14:03:50.322 INFO:tasks.rados.rados.0.smithi017.stdout:422: seq_num 216 ranges {430312=711668,1723287=549075,2781837=703343} 2023-03-29T14:03:50.328 INFO:tasks.rados.rados.0.smithi017.stdout:422: writing smithi017107884-48 from 430312 to 1141980 tid 1 2023-03-29T14:03:50.334 INFO:tasks.rados.rados.0.smithi017.stdout:422: writing smithi017107884-48 from 1723287 to 2272362 tid 2 2023-03-29T14:03:50.341 INFO:tasks.rados.rados.0.smithi017.stdout:422: writing smithi017107884-48 from 2781837 to 3485180 tid 3 2023-03-29T14:03:50.341 INFO:tasks.rados.rados.0.smithi017.stdout:423: write (excl) oid 12 current snap is 47 2023-03-29T14:03:50.341 INFO:tasks.rados.rados.0.smithi017.stdout:423: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2023-03-29T14:03:50.348 INFO:tasks.rados.rados.0.smithi017.stdout:423: writing smithi017107884-12 from 447119 to 1234036 tid 1 2023-03-29T14:03:50.354 INFO:tasks.rados.rados.0.smithi017.stdout:423: writing smithi017107884-12 from 1665416 to 2358149 tid 2 2023-03-29T14:03:50.360 INFO:tasks.rados.rados.0.smithi017.stdout:423: writing smithi017107884-12 from 2976554 to 3491169 tid 3 2023-03-29T14:03:50.361 INFO:tasks.rados.rados.0.smithi017.stdout:424: snap_remove snap 45 2023-03-29T14:03:50.408 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:50.410 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:50.422 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2023-03-29T14:03:50.301591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222931964,"num_objects":280,"num_object_clones":178,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1248,"num_read_kb":612155,"num_write":1726,"num_write_kb":423422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":549512,"kb_used_data":511912,"kb_used_omap":0,"kb_used_meta":37568,"kb_avail":376937848,"statfs":{"total":386547056640,"available":385984356352,"internally_reserved":0,"allocated":524197888,"data_stored":606746342,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":38469632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2117682,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":30051,"num_write":40,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014464"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":132,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.477730+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:45.477730+0000","last_peered":"2023-03-29T14:03:45.477730+0000","last_clean":"2023-03-29T14:03:45.477730+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:45.477730+0000","last_undegraded":"2023-03-29T14:03:45.477730+0000","last_fullsized":"2023-03-29T14:03:45.477730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":39299,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":96,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.307123+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:44.307123+0000","last_peered":"2023-03-29T14:03:44.307123+0000","last_clean":"2023-03-29T14:03:44.307123+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:44.307123+0000","last_undegraded":"2023-03-29T14:03:44.307123+0000","last_fullsized":"2023-03-29T14:03:44.307123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10764,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"100'75","reported_seq":241,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.312454+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:45.312454+0000","last_peered":"2023-03-29T14:03:45.312454+0000","last_clean":"2023-03-29T14:03:45.312454+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:45.312454+0000","last_undegraded":"2023-03-29T14:03:45.312454+0000","last_fullsized":"2023-03-29T14:03:45.312454+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16504474,"num_objects":24,"num_object_clones":14,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":80,"num_read_kb":40059,"num_write":154,"num_write_kb":44924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":200,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.298120+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:45.298120+0000","last_peered":"2023-03-29T14:03:45.298120+0000","last_clean":"2023-03-29T14:03:45.298120+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:45.298120+0000","last_undegraded":"2023-03-29T14:03:45.298120+0000","last_fullsized":"2023-03-29T14:03:45.298120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"93'55","reported_seq":211,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.304410+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:44.304410+0000","last_peered":"2023-03-29T14:03:44.304410+0000","last_clean":"2023-03-29T14:03:44.304410+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:44.304410+0000","last_undegraded":"2023-03-29T14:03:44.304410+0000","last_fullsized":"2023-03-29T14:03:44.304410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773386,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":59537,"num_write":114,"num_write_kb":33984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.307285+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:45.307285+0000","last_peered":"2023-03-29T14:03:45.307285+0000","last_clean":"2023-03-29T14:03:45.307285+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:45.307285+0000","last_undegraded":"2023-03-29T14:03:45.307285+0000","last_fullsized":"2023-03-29T14:03:45.307285+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"98'37","reported_seq":171,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.306198+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:45.306198+0000","last_peered":"2023-03-29T14:03:45.306198+0000","last_clean":"2023-03-29T14:03:45.306198+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:45.306198+0000","last_undegraded":"2023-03-29T14:03:45.306198+0000","last_fullsized":"2023-03-29T14:03:45.306198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17815354,"num_objects":16,"num_object_clones":10,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":38107,"num_write":78,"num_write_kb":21206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":159,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.293927+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:45.293927+0000","last_peered":"2023-03-29T14:03:45.293927+0000","last_clean":"2023-03-29T14:03:45.293927+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:45.293927+0000","last_undegraded":"2023-03-29T14:03:45.293927+0000","last_fullsized":"2023-03-29T14:03:45.293927+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.321897+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:44.321897+0000","last_peered":"2023-03-29T14:03:44.321897+0000","last_clean":"2023-03-29T14:03:44.321897+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:44.321897+0000","last_undegraded":"2023-03-29T14:03:44.321897+0000","last_fullsized":"2023-03-29T14:03:44.321897+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":125,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:46.439325+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:46.439325+0000","last_peered":"2023-03-29T14:03:46.439325+0000","last_clean":"2023-03-29T14:03:46.439325+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:46.439325+0000","last_undegraded":"2023-03-29T14:03:46.439325+0000","last_fullsized":"2023-03-29T14:03:46.439325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":33,"num_read_kb":13760,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.301408+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:45.301408+0000","last_peered":"2023-03-29T14:03:45.301408+0000","last_clean":"2023-03-29T14:03:45.301408+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:45.301408+0000","last_undegraded":"2023-03-29T14:03:45.301408+0000","last_fullsized":"2023-03-29T14:03:45.301408+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"98'31","reported_seq":152,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.331393+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:44.331393+0000","last_peered":"2023-03-29T14:03:44.331393+0000","last_clean":"2023-03-29T14:03:44.331393+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:44.331393+0000","last_undegraded":"2023-03-29T14:03:44.331393+0000","last_fullsized":"2023-03-29T14:03:44.331393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991712,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":52,"num_read_kb":31041,"num_write":64,"num_write_kb":16778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.475296+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:45.475296+0000","last_peered":"2023-03-29T14:03:45.475296+0000","last_clean":"2023-03-29T14:03:45.475296+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:45.475296+0000","last_undegraded":"2023-03-29T14:03:45.475296+0000","last_fullsized":"2023-03-29T14:03:45.475296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"100'48","reported_seq":193,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.543113+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:45.543113+0000","last_peered":"2023-03-29T14:03:45.543113+0000","last_clean":"2023-03-29T14:03:45.543113+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:45.543113+0000","last_undegraded":"2023-03-29T14:03:45.543113+0000","last_fullsized":"2023-03-29T14:03:45.543113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12470794,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":45585,"num_write":100,"num_write_kb":30334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.304355+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:45.304355+0000","last_peered":"2023-03-29T14:03:45.304355+0000","last_clean":"2023-03-29T14:03:45.304355+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:45.304355+0000","last_undegraded":"2023-03-29T14:03:45.304355+0000","last_fullsized":"2023-03-29T14:03:45.304355+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":304,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.288768+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:44.288768+0000","last_peered":"2023-03-29T14:03:44.288768+0000","last_clean":"2023-03-29T14:03:44.288768+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:44.288768+0000","last_undegraded":"2023-03-29T14:03:44.288768+0000","last_fullsized":"2023-03-29T14:03:44.288768+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":217,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.507795+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:45.507795+0000","last_peered":"2023-03-29T14:03:45.507795+0000","last_clean":"2023-03-29T14:03:45.507795+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:45.507795+0000","last_undegraded":"2023-03-29T14:03:45.507795+0000","last_fullsized":"2023-03-29T14:03:45.507795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":116,"num_read_kb":73205,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.297542+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:45.297542+0000","last_peered":"2023-03-29T14:03:45.297542+0000","last_clean":"2023-03-29T14:03:45.297542+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:45.297542+0000","last_undegraded":"2023-03-29T14:03:45.297542+0000","last_fullsized":"2023-03-29T14:03:45.297542+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"98'106","reported_seq":322,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.334954+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:44.334954+0000","last_peered":"2023-03-29T14:03:44.334954+0000","last_clean":"2023-03-29T14:03:44.334954+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:44.334954+0000","last_undegraded":"2023-03-29T14:03:44.334954+0000","last_fullsized":"2023-03-29T14:03:44.334954+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29109724,"num_objects":34,"num_object_clones":22,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":147,"num_read_kb":75809,"num_write":216,"num_write_kb":61498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.320728+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:44.320728+0000","last_peered":"2023-03-29T14:03:44.320728+0000","last_clean":"2023-03-29T14:03:44.320728+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:44.320728+0000","last_undegraded":"2023-03-29T14:03:44.320728+0000","last_fullsized":"2023-03-29T14:03:44.320728+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":162,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.327114+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:44.327114+0000","last_peered":"2023-03-29T14:03:44.327114+0000","last_clean":"2023-03-29T14:03:44.327114+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:44.327114+0000","last_undegraded":"2023-03-29T14:03:44.327114+0000","last_fullsized":"2023-03-29T14:03:44.327114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"100'57","reported_seq":205,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.671408+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:45.671408+0000","last_peered":"2023-03-29T14:03:45.671408+0000","last_clean":"2023-03-29T14:03:45.671408+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:45.671408+0000","last_undegraded":"2023-03-29T14:03:45.671408+0000","last_fullsized":"2023-03-29T14:03:45.671408+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27966782,"num_objects":24,"num_object_clones":14,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":71,"num_read_kb":34991,"num_write":124,"num_write_kb":37932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.324766+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:44.324766+0000","last_peered":"2023-03-29T14:03:44.324766+0000","last_clean":"2023-03-29T14:03:44.324766+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:44.324766+0000","last_undegraded":"2023-03-29T14:03:44.324766+0000","last_fullsized":"2023-03-29T14:03:44.324766+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-03-29T14:03:45.309196+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:45.309196+0000","last_peered":"2023-03-29T14:03:45.309196+0000","last_clean":"2023-03-29T14:03:45.309196+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:45.309196+0000","last_undegraded":"2023-03-29T14:03:45.309196+0000","last_fullsized":"2023-03-29T14:03:45.309196+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'51","reported_seq":203,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-03-29T14:03:44.304091+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:44.304091+0000","last_peered":"2023-03-29T14:03:44.304091+0000","last_clean":"2023-03-29T14:03:44.304091+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:44.304091+0000","last_undegraded":"2023-03-29T14:03:44.304091+0000","last_fullsized":"2023-03-29T14:03:44.304091+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20571702,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":45127,"num_write":106,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":221751228,"num_objects":276,"num_object_clones":178,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1106,"num_read_kb":612035,"num_write":1452,"num_write_kb":419634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":698,"ondisk_log_size":698,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":142312,"kb_used_data":135136,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94229528,"statfs":{"total":96636764160,"available":96491036672,"internally_reserved":0,"allocated":138379264,"data_stored":160616850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":98552,"kb_used_data":91632,"kb_used_omap":0,"kb_used_meta":6912,"kb_avail":94273288,"statfs":{"total":96636764160,"available":96535846912,"internally_reserved":0,"allocated":93831168,"data_stored":105233462,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7077888},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159312,"kb_used_data":147464,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94212528,"statfs":{"total":96636764160,"available":96473628672,"internally_reserved":0,"allocated":151003136,"data_stored":176629634,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705764,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149336,"kb_used_data":137680,"kb_used_omap":0,"kb_used_meta":11648,"kb_avail":94222504,"statfs":{"total":96636764160,"available":96483844096,"internally_reserved":0,"allocated":140984320,"data_stored":164266396,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":11927552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:51.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:51.309 INFO:tasks.rados.rados.0.smithi017.stdout:424: done (3 left) 2023-03-29T14:03:51.309 INFO:tasks.rados.rados.0.smithi017.stdout:425: snap_create 2023-03-29T14:03:51.309 INFO:tasks.rados.rados.0.smithi017.stdout:422: finishing write tid 1 to smithi017107884-48 2023-03-29T14:03:51.309 INFO:tasks.rados.rados.0.smithi017.stdout:421: finishing write tid 1 to smithi017107884-13 2023-03-29T14:03:51.309 INFO:tasks.rados.rados.0.smithi017.stdout:422: finishing write tid 2 to smithi017107884-48 2023-03-29T14:03:51.309 INFO:tasks.rados.rados.0.smithi017.stdout:422: finishing write tid 3 to smithi017107884-48 2023-03-29T14:03:51.309 INFO:tasks.rados.rados.0.smithi017.stdout:422: finishing write tid 4 to smithi017107884-48 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:422: finishing write tid 5 to smithi017107884-48 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:422: oid 48 updating version 0 to 80 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:422: oid 48 version 80 is already newer than 80 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:422: oid 48 version 80 is already newer than 80 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:422: oid 48 version 80 is already newer than 80 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 80 (ObjNum 216 snap 47 seq_num 216) dirty exists 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:422: left oid 48 (ObjNum 216 snap 47 seq_num 216) 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:421: finishing write tid 2 to smithi017107884-13 2023-03-29T14:03:51.310 INFO:tasks.rados.rados.0.smithi017.stdout:421: finishing write tid 3 to smithi017107884-13 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:421: finishing write tid 4 to smithi017107884-13 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:421: finishing write tid 5 to smithi017107884-13 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:421: oid 13 updating version 0 to 62 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:421: oid 13 version 62 is already newer than 62 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:421: oid 13 version 62 is already newer than 62 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:421: oid 13 version 62 is already newer than 58 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 62 (ObjNum 215 snap 47 seq_num 215) dirty exists 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:421: left oid 13 (ObjNum 215 snap 47 seq_num 215) 2023-03-29T14:03:51.311 INFO:tasks.rados.rados.0.smithi017.stdout:423: finishing write tid 1 to smithi017107884-12 2023-03-29T14:03:51.312 INFO:tasks.rados.rados.0.smithi017.stdout:423: finishing write tid 2 to smithi017107884-12 2023-03-29T14:03:51.312 INFO:tasks.rados.rados.0.smithi017.stdout:423: finishing write tid 3 to smithi017107884-12 2023-03-29T14:03:51.312 INFO:tasks.rados.rados.0.smithi017.stdout:423: finishing write tid 4 to smithi017107884-12 2023-03-29T14:03:51.312 INFO:tasks.rados.rados.0.smithi017.stdout:423: finishing write tid 5 to smithi017107884-12 2023-03-29T14:03:51.312 INFO:tasks.rados.rados.0.smithi017.stdout:423: oid 12 updating version 0 to 67 2023-03-29T14:03:51.312 INFO:tasks.rados.rados.0.smithi017.stdout:423: oid 12 version 67 is already newer than 67 2023-03-29T14:03:51.312 INFO:tasks.rados.rados.0.smithi017.stdout:423: oid 12 version 67 is already newer than 67 2023-03-29T14:03:51.313 INFO:tasks.rados.rados.0.smithi017.stdout:423: oid 12 version 67 is already newer than 63 2023-03-29T14:03:51.313 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 67 (ObjNum 217 snap 47 seq_num 217) dirty exists 2023-03-29T14:03:51.313 INFO:tasks.rados.rados.0.smithi017.stdout:423: left oid 12 (ObjNum 217 snap 47 seq_num 217) 2023-03-29T14:03:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:51.426+0000 7f441659f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:51.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:51.430+0000 7f441659f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:52.312 INFO:tasks.rados.rados.0.smithi017.stdout:421: done (3 left) 2023-03-29T14:03:52.313 INFO:tasks.rados.rados.0.smithi017.stdout:422: done (2 left) 2023-03-29T14:03:52.313 INFO:tasks.rados.rados.0.smithi017.stdout:423: done (1 left) 2023-03-29T14:03:52.313 INFO:tasks.rados.rados.0.smithi017.stdout:425: done (0 left) 2023-03-29T14:03:52.313 INFO:tasks.rados.rados.0.smithi017.stdout:426: snap_create 2023-03-29T14:03:53.188 INFO:tasks.rados.rados.0.smithi017.stdout:426: done (0 left) 2023-03-29T14:03:53.188 INFO:tasks.rados.rados.0.smithi017.stdout:427: write (excl) oid 14 current snap is 49 2023-03-29T14:03:53.188 INFO:tasks.rados.rados.0.smithi017.stdout:427: seq_num 218 ranges {463926=462166,1723898=436391,2603977=725887} 2023-03-29T14:03:53.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: writing smithi017107884-14 from 463926 to 926092 tid 1 2023-03-29T14:03:53.200 INFO:tasks.rados.rados.0.smithi017.stdout:427: writing smithi017107884-14 from 1723898 to 2160289 tid 2 2023-03-29T14:03:53.207 INFO:tasks.rados.rados.0.smithi017.stdout:427: writing smithi017107884-14 from 2603977 to 3329864 tid 3 2023-03-29T14:03:53.207 INFO:tasks.rados.rados.0.smithi017.stdout:428: write (excl) oid 17 current snap is 49 2023-03-29T14:03:53.207 INFO:tasks.rados.rados.0.smithi017.stdout:428: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2023-03-29T14:03:53.212 INFO:tasks.rados.rados.0.smithi017.stdout:428: writing smithi017107884-17 from 480733 to 1018148 tid 1 2023-03-29T14:03:53.221 INFO:tasks.rados.rados.0.smithi017.stdout:428: writing smithi017107884-17 from 1782380 to 2478782 tid 2 2023-03-29T14:03:53.227 INFO:tasks.rados.rados.0.smithi017.stdout:428: writing smithi017107884-17 from 3031400 to 3568559 tid 3 2023-03-29T14:03:53.227 INFO:tasks.rados.rados.0.smithi017.stdout:429: write oid 24 current snap is 49 2023-03-29T14:03:53.227 INFO:tasks.rados.rados.0.smithi017.stdout:429: seq_num 220 ranges {497540=612664,1840862=440060,3058823=464784} 2023-03-29T14:03:53.233 INFO:tasks.rados.rados.0.smithi017.stdout:429: writing smithi017107884-24 from 497540 to 1110204 tid 1 2023-03-29T14:03:53.239 INFO:tasks.rados.rados.0.smithi017.stdout:429: writing smithi017107884-24 from 1840862 to 2280922 tid 2 2023-03-29T14:03:53.246 INFO:tasks.rados.rados.0.smithi017.stdout:429: writing smithi017107884-24 from 3058823 to 3523607 tid 3 2023-03-29T14:03:53.246 INFO:tasks.rados.rados.0.smithi017.stdout:430: snap_remove snap 41 2023-03-29T14:03:53.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:53.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:53.635+0000 7efe62b48700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:53.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:53.640+0000 7efe62b48700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:430: done (3 left) 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:431: snap_create 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: finishing write tid 1 to smithi017107884-14 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: finishing write tid 2 to smithi017107884-14 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: finishing write tid 3 to smithi017107884-14 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:428: finishing write tid 1 to smithi017107884-17 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: finishing write tid 4 to smithi017107884-14 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: finishing write tid 5 to smithi017107884-14 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: oid 14 updating version 0 to 43 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: oid 14 updating version 43 to 47 2023-03-29T14:03:54.193 INFO:tasks.rados.rados.0.smithi017.stdout:427: oid 14 version 47 is already newer than 47 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:427: oid 14 version 47 is already newer than 47 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 47 (ObjNum 218 snap 49 seq_num 218) dirty exists 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:427: left oid 14 (ObjNum 218 snap 49 seq_num 218) 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:429: finishing write tid 1 to smithi017107884-24 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:429: finishing write tid 2 to smithi017107884-24 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: finishing write tid 2 to smithi017107884-17 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: finishing write tid 3 to smithi017107884-17 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: finishing write tid 4 to smithi017107884-17 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: finishing write tid 5 to smithi017107884-17 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: oid 17 updating version 0 to 85 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: oid 17 version 85 is already newer than 85 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: oid 17 version 85 is already newer than 81 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: oid 17 version 85 is already newer than 85 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 85 (ObjNum 219 snap 49 seq_num 219) dirty exists 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: left oid 17 (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:429: finishing write tid 3 to smithi017107884-24 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:429: finishing write tid 4 to smithi017107884-24 2023-03-29T14:03:54.194 INFO:tasks.rados.rados.0.smithi017.stdout:429: finishing write tid 5 to smithi017107884-24 2023-03-29T14:03:54.195 INFO:tasks.rados.rados.0.smithi017.stdout:429: oid 24 updating version 0 to 36 2023-03-29T14:03:54.195 INFO:tasks.rados.rados.0.smithi017.stdout:429: oid 24 version 36 is already newer than 36 2023-03-29T14:03:54.195 INFO:tasks.rados.rados.0.smithi017.stdout:429: oid 24 version 36 is already newer than 36 2023-03-29T14:03:54.195 INFO:tasks.rados.rados.0.smithi017.stdout:429: oid 24 version 36 is already newer than 36 2023-03-29T14:03:54.195 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 36 (ObjNum 220 snap 49 seq_num 220) dirty exists 2023-03-29T14:03:54.195 INFO:tasks.rados.rados.0.smithi017.stdout:429: left oid 24 (ObjNum 220 snap 49 seq_num 220) 2023-03-29T14:03:55.194 INFO:tasks.rados.rados.0.smithi017.stdout:427: done (3 left) 2023-03-29T14:03:55.194 INFO:tasks.rados.rados.0.smithi017.stdout:428: done (2 left) 2023-03-29T14:03:55.194 INFO:tasks.rados.rados.0.smithi017.stdout:429: done (1 left) 2023-03-29T14:03:55.194 INFO:tasks.rados.rados.0.smithi017.stdout:431: done (0 left) 2023-03-29T14:03:55.194 INFO:tasks.rados.rados.0.smithi017.stdout:432: delete oid 12 current snap is 50 2023-03-29T14:03:55.284 INFO:tasks.rados.rados.0.smithi017.stdout:432: done (0 left) 2023-03-29T14:03:55.284 INFO:tasks.rados.rados.0.smithi017.stdout:433: write (excl) oid 43 current snap is 50 2023-03-29T14:03:55.284 INFO:tasks.rados.rados.0.smithi017.stdout:433: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2023-03-29T14:03:55.289 INFO:tasks.rados.rados.0.smithi017.stdout:433: writing smithi017107884-43 from 531154 to 1010669 tid 1 2023-03-29T14:03:55.295 INFO:tasks.rados.rados.0.smithi017.stdout:433: writing smithi017107884-43 from 1557826 to 2001555 tid 2 2023-03-29T14:03:55.301 INFO:tasks.rados.rados.0.smithi017.stdout:433: writing smithi017107884-43 from 2713669 to 3200997 tid 3 2023-03-29T14:03:55.304 INFO:tasks.rados.rados.0.smithi017.stdout:433: writing smithi017107884-43 from 3679295 to 3731154 tid 4 2023-03-29T14:03:55.304 INFO:tasks.rados.rados.0.smithi017.stdout:434: snap_create 2023-03-29T14:03:55.338 INFO:tasks.rados.rados.0.smithi017.stdout:433: finishing write tid 1 to smithi017107884-43 2023-03-29T14:03:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:433: finishing write tid 2 to smithi017107884-43 2023-03-29T14:03:55.379 INFO:tasks.rados.rados.0.smithi017.stdout:433: finishing write tid 3 to smithi017107884-43 2023-03-29T14:03:55.380 INFO:tasks.rados.rados.0.smithi017.stdout:433: finishing write tid 4 to smithi017107884-43 2023-03-29T14:03:55.381 INFO:tasks.rados.rados.0.smithi017.stdout:433: finishing write tid 5 to smithi017107884-43 2023-03-29T14:03:55.382 INFO:tasks.rados.rados.0.smithi017.stdout:433: finishing write tid 6 to smithi017107884-43 2023-03-29T14:03:55.382 INFO:tasks.rados.rados.0.smithi017.stdout:433: oid 43 updating version 0 to 53 2023-03-29T14:03:55.382 INFO:tasks.rados.rados.0.smithi017.stdout:433: oid 43 version 53 is already newer than 49 2023-03-29T14:03:55.383 INFO:tasks.rados.rados.0.smithi017.stdout:433: oid 43 version 53 is already newer than 53 2023-03-29T14:03:55.383 INFO:tasks.rados.rados.0.smithi017.stdout:433: oid 43 version 53 is already newer than 53 2023-03-29T14:03:55.383 INFO:tasks.rados.rados.0.smithi017.stdout:433: oid 43 version 53 is already newer than 53 2023-03-29T14:03:55.383 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 53 (ObjNum 222 snap 50 seq_num 222) dirty exists 2023-03-29T14:03:55.383 INFO:tasks.rados.rados.0.smithi017.stdout:433: left oid 43 (ObjNum 222 snap 50 seq_num 222) 2023-03-29T14:03:55.426 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:03:55.426 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:03:55.561 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:55.555+0000 7fef71a7f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:55.565 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:55.564+0000 7fef71a7f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:55.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:03:55.840 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:55.837+0000 7fb611656700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:55.842+0000 7fb611656700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:56.194 INFO:tasks.rados.rados.0.smithi017.stdout:433: done (1 left) 2023-03-29T14:03:56.195 INFO:tasks.rados.rados.0.smithi017.stdout:434: done (0 left) 2023-03-29T14:03:56.195 INFO:tasks.rados.rados.0.smithi017.stdout:435: snap_create 2023-03-29T14:03:56.199 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:03:56.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:03:56.346 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:56.340+0000 7fbf771dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:56.349 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:56.348+0000 7fbf771dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:57.195 INFO:tasks.rados.rados.0.smithi017.stdout:435: done (0 left) 2023-03-29T14:03:57.196 INFO:tasks.rados.rados.0.smithi017.stdout:436: write oid 45 current snap is 52 2023-03-29T14:03:57.196 INFO:tasks.rados.rados.0.smithi017.stdout:436: seq_num 223 ranges {547961=554764,1616308=703740,2741092=698600} 2023-03-29T14:03:57.199 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:03:57.204 INFO:tasks.rados.rados.0.smithi017.stdout:436: writing smithi017107884-45 from 547961 to 1102725 tid 1 2023-03-29T14:03:57.214 INFO:tasks.rados.rados.0.smithi017.stdout:436: writing smithi017107884-45 from 1616308 to 2320048 tid 2 2023-03-29T14:03:57.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:03:57.220 INFO:tasks.rados.rados.0.smithi017.stdout:436: writing smithi017107884-45 from 2741092 to 3439692 tid 3 2023-03-29T14:03:57.220 INFO:tasks.rados.rados.0.smithi017.stdout:437: snap_create 2023-03-29T14:03:57.319 INFO:tasks.rados.rados.0.smithi017.stdout:436: finishing write tid 1 to smithi017107884-45 2023-03-29T14:03:57.330 INFO:tasks.rados.rados.0.smithi017.stdout:436: finishing write tid 2 to smithi017107884-45 2023-03-29T14:03:57.331 INFO:tasks.rados.rados.0.smithi017.stdout:436: finishing write tid 3 to smithi017107884-45 2023-03-29T14:03:57.332 INFO:tasks.rados.rados.0.smithi017.stdout:436: finishing write tid 4 to smithi017107884-45 2023-03-29T14:03:57.333 INFO:tasks.rados.rados.0.smithi017.stdout:436: finishing write tid 5 to smithi017107884-45 2023-03-29T14:03:57.333 INFO:tasks.rados.rados.0.smithi017.stdout:436: oid 45 updating version 0 to 60 2023-03-29T14:03:57.333 INFO:tasks.rados.rados.0.smithi017.stdout:436: oid 45 version 60 is already newer than 60 2023-03-29T14:03:57.334 INFO:tasks.rados.rados.0.smithi017.stdout:436: oid 45 version 60 is already newer than 60 2023-03-29T14:03:57.334 INFO:tasks.rados.rados.0.smithi017.stdout:436: oid 45 version 60 is already newer than 58 2023-03-29T14:03:57.334 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 60 (ObjNum 223 snap 52 seq_num 223) dirty exists 2023-03-29T14:03:57.334 INFO:tasks.rados.rados.0.smithi017.stdout:436: left oid 45 (ObjNum 223 snap 52 seq_num 223) 2023-03-29T14:03:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:57.357+0000 7f3b44d71700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:57.366 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:57.365+0000 7f3b44d71700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:57.912 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:03:58.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:58.041+0000 7f3667b28700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:58.046 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:58.045+0000 7f3667b28700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:58.196 INFO:tasks.rados.rados.0.smithi017.stdout:436: done (1 left) 2023-03-29T14:03:58.196 INFO:tasks.rados.rados.0.smithi017.stdout:437: done (0 left) 2023-03-29T14:03:58.196 INFO:tasks.rados.rados.0.smithi017.stdout:438: snap_create 2023-03-29T14:03:58.198 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:03:58.210 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:03:58.349 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:58.337+0000 7fb7e5f30700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:58.350 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:58.345+0000 7fb7e5f30700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:59.197 INFO:tasks.rados.rados.0.smithi017.stdout:438: done (0 left) 2023-03-29T14:03:59.198 INFO:tasks.rados.rados.0.smithi017.stdout:439: delete oid 13 current snap is 54 2023-03-29T14:03:59.200 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:03:59.214 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:03:59.214 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:59.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:59.346+0000 7ffb45eba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:59.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:59.353+0000 7ffb45eba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:59.403 INFO:tasks.rados.rados.0.smithi017.stdout:439: done (0 left) 2023-03-29T14:03:59.403 INFO:tasks.rados.rados.0.smithi017.stdout:440: write (excl) oid 36 current snap is 54 2023-03-29T14:03:59.404 INFO:tasks.rados.rados.0.smithi017.stdout:440: seq_num 225 ranges {581575=705262,1733272=707409,3195938=437497} 2023-03-29T14:03:59.410 INFO:tasks.rados.rados.0.smithi017.stdout:440: writing smithi017107884-36 from 581575 to 1286837 tid 1 2023-03-29T14:03:59.417 INFO:tasks.rados.rados.0.smithi017.stdout:440: writing smithi017107884-36 from 1733272 to 2440681 tid 2 2023-03-29T14:03:59.424 INFO:tasks.rados.rados.0.smithi017.stdout:440: writing smithi017107884-36 from 3195938 to 3633435 tid 3 2023-03-29T14:03:59.424 INFO:tasks.rados.rados.0.smithi017.stdout:441: snap_create 2023-03-29T14:03:59.469 INFO:tasks.rados.rados.0.smithi017.stdout:440: finishing write tid 1 to smithi017107884-36 2023-03-29T14:03:59.509 INFO:tasks.rados.rados.0.smithi017.stdout:440: finishing write tid 2 to smithi017107884-36 2023-03-29T14:03:59.541 INFO:tasks.rados.rados.0.smithi017.stdout:440: finishing write tid 3 to smithi017107884-36 2023-03-29T14:03:59.542 INFO:tasks.rados.rados.0.smithi017.stdout:440: finishing write tid 4 to smithi017107884-36 2023-03-29T14:03:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:440: finishing write tid 5 to smithi017107884-36 2023-03-29T14:03:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:440: oid 36 updating version 0 to 58 2023-03-29T14:03:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:440: oid 36 version 58 is already newer than 58 2023-03-29T14:03:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:440: oid 36 version 58 is already newer than 58 2023-03-29T14:03:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:440: oid 36 version 58 is already newer than 54 2023-03-29T14:03:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 58 (ObjNum 225 snap 54 seq_num 225) dirty exists 2023-03-29T14:03:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:440: left oid 36 (ObjNum 225 snap 54 seq_num 225) 2023-03-29T14:03:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:59.563 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:59.575 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-03-29T14:03:59.196320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235570828,"num_objects":298,"num_object_clones":194,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1285,"num_read_kb":628489,"num_write":1818,"num_write_kb":452932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":597956,"kb_used_data":559588,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":376889404,"statfs":{"total":386547056640,"available":385934749696,"internally_reserved":0,"allocated":573018112,"data_stored":672376214,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1344560,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":42,"num_write_kb":12464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.003908"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":145,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.208699+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:56.208699+0000","last_peered":"2023-03-29T14:03:56.208699+0000","last_clean":"2023-03-29T14:03:56.208699+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:56.208699+0000","last_undegraded":"2023-03-29T14:03:56.208699+0000","last_fullsized":"2023-03-29T14:03:56.208699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":102,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.337222+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:54.337222+0000","last_peered":"2023-03-29T14:03:54.337222+0000","last_clean":"2023-03-29T14:03:54.337222+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:54.337222+0000","last_undegraded":"2023-03-29T14:03:54.337222+0000","last_fullsized":"2023-03-29T14:03:54.337222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10764,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'85","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.558036+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:53.558036+0000","last_peered":"2023-03-29T14:03:53.558036+0000","last_clean":"2023-03-29T14:03:53.558036+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:53.558036+0000","last_undegraded":"2023-03-29T14:03:53.558036+0000","last_fullsized":"2023-03-29T14:03:53.558036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529734,"num_objects":28,"num_object_clones":18,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":84,"num_read_kb":40061,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":206,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.260329+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:53.260329+0000","last_peered":"2023-03-29T14:03:53.260329+0000","last_clean":"2023-03-29T14:03:53.260329+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:53.260329+0000","last_undegraded":"2023-03-29T14:03:53.260329+0000","last_fullsized":"2023-03-29T14:03:53.260329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"93'55","reported_seq":217,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.352874+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:54.352874+0000","last_peered":"2023-03-29T14:03:54.352874+0000","last_clean":"2023-03-29T14:03:54.352874+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:54.352874+0000","last_undegraded":"2023-03-29T14:03:54.352874+0000","last_fullsized":"2023-03-29T14:03:54.352874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773386,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":59537,"num_write":114,"num_write_kb":33984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.259185+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:53.259185+0000","last_peered":"2023-03-29T14:03:53.259185+0000","last_clean":"2023-03-29T14:03:53.259185+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:53.259185+0000","last_undegraded":"2023-03-29T14:03:53.259185+0000","last_fullsized":"2023-03-29T14:03:53.259185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":195,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.494797+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:53.494797+0000","last_peered":"2023-03-29T14:03:53.494797+0000","last_clean":"2023-03-29T14:03:53.494797+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:53.494797+0000","last_undegraded":"2023-03-29T14:03:53.494797+0000","last_fullsized":"2023-03-29T14:03:53.494797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":38109,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":165,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.258490+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:53.258490+0000","last_peered":"2023-03-29T14:03:53.258490+0000","last_clean":"2023-03-29T14:03:53.258490+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:53.258490+0000","last_undegraded":"2023-03-29T14:03:53.258490+0000","last_fullsized":"2023-03-29T14:03:53.258490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.353696+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:54.353696+0000","last_peered":"2023-03-29T14:03:54.353696+0000","last_clean":"2023-03-29T14:03:54.353696+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:54.353696+0000","last_undegraded":"2023-03-29T14:03:54.353696+0000","last_fullsized":"2023-03-29T14:03:54.353696+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":138,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.215412+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:56.215412+0000","last_peered":"2023-03-29T14:03:56.215412+0000","last_clean":"2023-03-29T14:03:56.215412+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:56.215412+0000","last_undegraded":"2023-03-29T14:03:56.215412+0000","last_fullsized":"2023-03-29T14:03:56.215412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.242602+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:53.242602+0000","last_peered":"2023-03-29T14:03:53.242602+0000","last_clean":"2023-03-29T14:03:53.242602+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:53.242602+0000","last_undegraded":"2023-03-29T14:03:53.242602+0000","last_fullsized":"2023-03-29T14:03:53.242602+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"108'36","reported_seq":168,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.572964+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:53.572964+0000","last_peered":"2023-03-29T14:03:53.572964+0000","last_clean":"2023-03-29T14:03:53.572964+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:53.572964+0000","last_undegraded":"2023-03-29T14:03:53.572964+0000","last_fullsized":"2023-03-29T14:03:53.572964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7395080,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":31042,"num_write":74,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.219419+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:56.219419+0000","last_peered":"2023-03-29T14:03:56.219419+0000","last_clean":"2023-03-29T14:03:56.219419+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:56.219419+0000","last_undegraded":"2023-03-29T14:03:56.219419+0000","last_fullsized":"2023-03-29T14:03:56.219419+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"110'53","reported_seq":210,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:55.372310+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:55.372310+0000","last_peered":"2023-03-29T14:03:55.372310+0000","last_clean":"2023-03-29T14:03:55.372310+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:55.372310+0000","last_undegraded":"2023-03-29T14:03:55.372310+0000","last_fullsized":"2023-03-29T14:03:55.372310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19933102,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":45586,"num_write":112,"num_write_kb":33194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.248519+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:53.248519+0000","last_peered":"2023-03-29T14:03:53.248519+0000","last_clean":"2023-03-29T14:03:53.248519+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:53.248519+0000","last_undegraded":"2023-03-29T14:03:53.248519+0000","last_fullsized":"2023-03-29T14:03:53.248519+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":310,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.335885+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:54.335885+0000","last_peered":"2023-03-29T14:03:54.335885+0000","last_clean":"2023-03-29T14:03:54.335885+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:54.335885+0000","last_undegraded":"2023-03-29T14:03:54.335885+0000","last_fullsized":"2023-03-29T14:03:54.335885+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":223,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.244344+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:53.244344+0000","last_peered":"2023-03-29T14:03:53.244344+0000","last_clean":"2023-03-29T14:03:53.244344+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:53.244344+0000","last_undegraded":"2023-03-29T14:03:53.244344+0000","last_fullsized":"2023-03-29T14:03:53.244344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":116,"num_read_kb":73205,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.247973+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:53.247973+0000","last_peered":"2023-03-29T14:03:53.247973+0000","last_clean":"2023-03-29T14:03:53.247973+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:53.247973+0000","last_undegraded":"2023-03-29T14:03:53.247973+0000","last_fullsized":"2023-03-29T14:03:53.247973+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":338,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.406769+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:53.406769+0000","last_peered":"2023-03-29T14:03:53.406769+0000","last_clean":"2023-03-29T14:03:53.406769+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:53.406769+0000","last_undegraded":"2023-03-29T14:03:53.406769+0000","last_fullsized":"2023-03-29T14:03:53.406769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":148,"num_read_kb":75810,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.330293+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:54.330293+0000","last_peered":"2023-03-29T14:03:54.330293+0000","last_clean":"2023-03-29T14:03:54.330293+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:54.330293+0000","last_undegraded":"2023-03-29T14:03:54.330293+0000","last_fullsized":"2023-03-29T14:03:54.330293+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":168,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.338422+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:54.338422+0000","last_peered":"2023-03-29T14:03:54.338422+0000","last_clean":"2023-03-29T14:03:54.338422+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:54.338422+0000","last_undegraded":"2023-03-29T14:03:54.338422+0000","last_fullsized":"2023-03-29T14:03:54.338422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"105'67","reported_seq":231,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:55.240129+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:55.240129+0000","last_peered":"2023-03-29T14:03:55.240129+0000","last_clean":"2023-03-29T14:03:55.240129+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:55.240129+0000","last_undegraded":"2023-03-29T14:03:55.240129+0000","last_fullsized":"2023-03-29T14:03:55.240129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479280,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":79,"num_read_kb":34993,"num_write":144,"num_write_kb":45382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.351474+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:54.351474+0000","last_peered":"2023-03-29T14:03:54.351474+0000","last_clean":"2023-03-29T14:03:54.351474+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:54.351474+0000","last_undegraded":"2023-03-29T14:03:54.351474+0000","last_fullsized":"2023-03-29T14:03:54.351474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.252180+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:53.252180+0000","last_peered":"2023-03-29T14:03:53.252180+0000","last_clean":"2023-03-29T14:03:53.252180+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:53.252180+0000","last_undegraded":"2023-03-29T14:03:53.252180+0000","last_fullsized":"2023-03-29T14:03:53.252180+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'51","reported_seq":210,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.378284+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:53.378284+0000","last_peered":"2023-03-29T14:03:53.378284+0000","last_clean":"2023-03-29T14:03:53.378284+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:53.378284+0000","last_undegraded":"2023-03-29T14:03:53.378284+0000","last_fullsized":"2023-03-29T14:03:53.378284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20571702,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":45127,"num_write":106,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":234390092,"num_objects":294,"num_object_clones":194,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1143,"num_read_kb":628369,"num_write":1544,"num_write_kb":449144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":156196,"kb_used_data":148828,"kb_used_omap":0,"kb_used_meta":7360,"kb_avail":94215644,"statfs":{"total":96636764160,"available":96476819456,"internally_reserved":0,"allocated":152399872,"data_stored":179162625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7536640},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106160,"kb_used_data":99112,"kb_used_omap":0,"kb_used_meta":7040,"kb_avail":94265680,"statfs":{"total":96636764160,"available":96528056320,"internally_reserved":0,"allocated":101490688,"data_stored":115269114,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7208960},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":174176,"kb_used_data":162072,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94197664,"statfs":{"total":96636764160,"available":96458407936,"internally_reserved":0,"allocated":165961728,"data_stored":196445313,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":161424,"kb_used_data":149576,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94210416,"statfs":{"total":96636764160,"available":96471465984,"internally_reserved":0,"allocated":153165824,"data_stored":181499162,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:59.580 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:03:59.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:59.707+0000 7f94130d2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:59.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:03:59.714+0000 7f94130d2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:03:59.918 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:03:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:03:59.933 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-03-29T14:03:59.196320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235570828,"num_objects":298,"num_object_clones":194,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1285,"num_read_kb":628489,"num_write":1818,"num_write_kb":452932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":597956,"kb_used_data":559588,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":376889404,"statfs":{"total":386547056640,"available":385934749696,"internally_reserved":0,"allocated":573018112,"data_stored":672376214,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1344560,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":42,"num_write_kb":12464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.003908"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":145,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.208699+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:56.208699+0000","last_peered":"2023-03-29T14:03:56.208699+0000","last_clean":"2023-03-29T14:03:56.208699+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:56.208699+0000","last_undegraded":"2023-03-29T14:03:56.208699+0000","last_fullsized":"2023-03-29T14:03:56.208699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":102,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.337222+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:54.337222+0000","last_peered":"2023-03-29T14:03:54.337222+0000","last_clean":"2023-03-29T14:03:54.337222+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:54.337222+0000","last_undegraded":"2023-03-29T14:03:54.337222+0000","last_fullsized":"2023-03-29T14:03:54.337222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10764,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'85","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.558036+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:53.558036+0000","last_peered":"2023-03-29T14:03:53.558036+0000","last_clean":"2023-03-29T14:03:53.558036+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:53.558036+0000","last_undegraded":"2023-03-29T14:03:53.558036+0000","last_fullsized":"2023-03-29T14:03:53.558036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529734,"num_objects":28,"num_object_clones":18,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":84,"num_read_kb":40061,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":206,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.260329+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:53.260329+0000","last_peered":"2023-03-29T14:03:53.260329+0000","last_clean":"2023-03-29T14:03:53.260329+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:53.260329+0000","last_undegraded":"2023-03-29T14:03:53.260329+0000","last_fullsized":"2023-03-29T14:03:53.260329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"93'55","reported_seq":217,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.352874+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:54.352874+0000","last_peered":"2023-03-29T14:03:54.352874+0000","last_clean":"2023-03-29T14:03:54.352874+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:54.352874+0000","last_undegraded":"2023-03-29T14:03:54.352874+0000","last_fullsized":"2023-03-29T14:03:54.352874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773386,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":59537,"num_write":114,"num_write_kb":33984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.259185+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:53.259185+0000","last_peered":"2023-03-29T14:03:53.259185+0000","last_clean":"2023-03-29T14:03:53.259185+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:53.259185+0000","last_undegraded":"2023-03-29T14:03:53.259185+0000","last_fullsized":"2023-03-29T14:03:53.259185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":195,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.494797+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:53.494797+0000","last_peered":"2023-03-29T14:03:53.494797+0000","last_clean":"2023-03-29T14:03:53.494797+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:53.494797+0000","last_undegraded":"2023-03-29T14:03:53.494797+0000","last_fullsized":"2023-03-29T14:03:53.494797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":38109,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":165,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.258490+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:53.258490+0000","last_peered":"2023-03-29T14:03:53.258490+0000","last_clean":"2023-03-29T14:03:53.258490+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:53.258490+0000","last_undegraded":"2023-03-29T14:03:53.258490+0000","last_fullsized":"2023-03-29T14:03:53.258490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.353696+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:54.353696+0000","last_peered":"2023-03-29T14:03:54.353696+0000","last_clean":"2023-03-29T14:03:54.353696+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:54.353696+0000","last_undegraded":"2023-03-29T14:03:54.353696+0000","last_fullsized":"2023-03-29T14:03:54.353696+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":138,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.215412+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:56.215412+0000","last_peered":"2023-03-29T14:03:56.215412+0000","last_clean":"2023-03-29T14:03:56.215412+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:56.215412+0000","last_undegraded":"2023-03-29T14:03:56.215412+0000","last_fullsized":"2023-03-29T14:03:56.215412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.242602+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:53.242602+0000","last_peered":"2023-03-29T14:03:53.242602+0000","last_clean":"2023-03-29T14:03:53.242602+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:53.242602+0000","last_undegraded":"2023-03-29T14:03:53.242602+0000","last_fullsized":"2023-03-29T14:03:53.242602+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"108'36","reported_seq":168,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.572964+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:53.572964+0000","last_peered":"2023-03-29T14:03:53.572964+0000","last_clean":"2023-03-29T14:03:53.572964+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:53.572964+0000","last_undegraded":"2023-03-29T14:03:53.572964+0000","last_fullsized":"2023-03-29T14:03:53.572964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7395080,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":31042,"num_write":74,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.219419+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:56.219419+0000","last_peered":"2023-03-29T14:03:56.219419+0000","last_clean":"2023-03-29T14:03:56.219419+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:56.219419+0000","last_undegraded":"2023-03-29T14:03:56.219419+0000","last_fullsized":"2023-03-29T14:03:56.219419+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"110'53","reported_seq":210,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:55.372310+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:55.372310+0000","last_peered":"2023-03-29T14:03:55.372310+0000","last_clean":"2023-03-29T14:03:55.372310+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:55.372310+0000","last_undegraded":"2023-03-29T14:03:55.372310+0000","last_fullsized":"2023-03-29T14:03:55.372310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19933102,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":45586,"num_write":112,"num_write_kb":33194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.248519+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:53.248519+0000","last_peered":"2023-03-29T14:03:53.248519+0000","last_clean":"2023-03-29T14:03:53.248519+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:53.248519+0000","last_undegraded":"2023-03-29T14:03:53.248519+0000","last_fullsized":"2023-03-29T14:03:53.248519+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":310,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.335885+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:54.335885+0000","last_peered":"2023-03-29T14:03:54.335885+0000","last_clean":"2023-03-29T14:03:54.335885+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:54.335885+0000","last_undegraded":"2023-03-29T14:03:54.335885+0000","last_fullsized":"2023-03-29T14:03:54.335885+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":223,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.244344+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:53.244344+0000","last_peered":"2023-03-29T14:03:53.244344+0000","last_clean":"2023-03-29T14:03:53.244344+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:53.244344+0000","last_undegraded":"2023-03-29T14:03:53.244344+0000","last_fullsized":"2023-03-29T14:03:53.244344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":116,"num_read_kb":73205,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.247973+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:53.247973+0000","last_peered":"2023-03-29T14:03:53.247973+0000","last_clean":"2023-03-29T14:03:53.247973+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:53.247973+0000","last_undegraded":"2023-03-29T14:03:53.247973+0000","last_fullsized":"2023-03-29T14:03:53.247973+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":338,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.406769+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:53.406769+0000","last_peered":"2023-03-29T14:03:53.406769+0000","last_clean":"2023-03-29T14:03:53.406769+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:53.406769+0000","last_undegraded":"2023-03-29T14:03:53.406769+0000","last_fullsized":"2023-03-29T14:03:53.406769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":148,"num_read_kb":75810,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.330293+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:54.330293+0000","last_peered":"2023-03-29T14:03:54.330293+0000","last_clean":"2023-03-29T14:03:54.330293+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:54.330293+0000","last_undegraded":"2023-03-29T14:03:54.330293+0000","last_fullsized":"2023-03-29T14:03:54.330293+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":168,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.338422+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:54.338422+0000","last_peered":"2023-03-29T14:03:54.338422+0000","last_clean":"2023-03-29T14:03:54.338422+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:54.338422+0000","last_undegraded":"2023-03-29T14:03:54.338422+0000","last_fullsized":"2023-03-29T14:03:54.338422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"105'67","reported_seq":231,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:55.240129+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:55.240129+0000","last_peered":"2023-03-29T14:03:55.240129+0000","last_clean":"2023-03-29T14:03:55.240129+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:55.240129+0000","last_undegraded":"2023-03-29T14:03:55.240129+0000","last_fullsized":"2023-03-29T14:03:55.240129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479280,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":79,"num_read_kb":34993,"num_write":144,"num_write_kb":45382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.351474+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:54.351474+0000","last_peered":"2023-03-29T14:03:54.351474+0000","last_clean":"2023-03-29T14:03:54.351474+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:54.351474+0000","last_undegraded":"2023-03-29T14:03:54.351474+0000","last_fullsized":"2023-03-29T14:03:54.351474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.252180+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:53.252180+0000","last_peered":"2023-03-29T14:03:53.252180+0000","last_clean":"2023-03-29T14:03:53.252180+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:53.252180+0000","last_undegraded":"2023-03-29T14:03:53.252180+0000","last_fullsized":"2023-03-29T14:03:53.252180+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'51","reported_seq":210,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.378284+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:53.378284+0000","last_peered":"2023-03-29T14:03:53.378284+0000","last_clean":"2023-03-29T14:03:53.378284+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:53.378284+0000","last_undegraded":"2023-03-29T14:03:53.378284+0000","last_fullsized":"2023-03-29T14:03:53.378284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20571702,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":45127,"num_write":106,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":234390092,"num_objects":294,"num_object_clones":194,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1143,"num_read_kb":628369,"num_write":1544,"num_write_kb":449144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":156196,"kb_used_data":148828,"kb_used_omap":0,"kb_used_meta":7360,"kb_avail":94215644,"statfs":{"total":96636764160,"available":96476819456,"internally_reserved":0,"allocated":152399872,"data_stored":179162625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7536640},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106160,"kb_used_data":99112,"kb_used_omap":0,"kb_used_meta":7040,"kb_avail":94265680,"statfs":{"total":96636764160,"available":96528056320,"internally_reserved":0,"allocated":101490688,"data_stored":115269114,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7208960},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":174176,"kb_used_data":162072,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94197664,"statfs":{"total":96636764160,"available":96458407936,"internally_reserved":0,"allocated":165961728,"data_stored":196445313,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":161424,"kb_used_data":149576,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94210416,"statfs":{"total":96636764160,"available":96471465984,"internally_reserved":0,"allocated":153165824,"data_stored":181499162,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:03:59.937 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:03:59.937 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:03:59.937 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:00.065+0000 7fba4b7ea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:00.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:00.071+0000 7fba4b7ea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:00.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:00.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:00.246+0000 7f2488166700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:00.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:00.250+0000 7f2488166700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:00.280 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:00.281 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:00.293 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-03-29T14:04:00.117005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235570828,"num_objects":298,"num_object_clones":194,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1285,"num_read_kb":628489,"num_write":1818,"num_write_kb":452932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":597956,"kb_used_data":559588,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":376889404,"statfs":{"total":386547056640,"available":385934749696,"internally_reserved":0,"allocated":573018112,"data_stored":672376214,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1344560,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":4,"num_write":42,"num_write_kb":12464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000763"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":145,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.208699+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:56.208699+0000","last_peered":"2023-03-29T14:03:56.208699+0000","last_clean":"2023-03-29T14:03:56.208699+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:56.208699+0000","last_undegraded":"2023-03-29T14:03:56.208699+0000","last_fullsized":"2023-03-29T14:03:56.208699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":102,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.337222+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:54.337222+0000","last_peered":"2023-03-29T14:03:54.337222+0000","last_clean":"2023-03-29T14:03:54.337222+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:54.337222+0000","last_undegraded":"2023-03-29T14:03:54.337222+0000","last_fullsized":"2023-03-29T14:03:54.337222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10764,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'85","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.558036+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:03:53.558036+0000","last_peered":"2023-03-29T14:03:53.558036+0000","last_clean":"2023-03-29T14:03:53.558036+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:03:53.558036+0000","last_undegraded":"2023-03-29T14:03:53.558036+0000","last_fullsized":"2023-03-29T14:03:53.558036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529734,"num_objects":28,"num_object_clones":18,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":84,"num_read_kb":40061,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":206,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.260329+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:03:53.260329+0000","last_peered":"2023-03-29T14:03:53.260329+0000","last_clean":"2023-03-29T14:03:53.260329+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:03:53.260329+0000","last_undegraded":"2023-03-29T14:03:53.260329+0000","last_fullsized":"2023-03-29T14:03:53.260329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"93'55","reported_seq":217,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.352874+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:54.352874+0000","last_peered":"2023-03-29T14:03:54.352874+0000","last_clean":"2023-03-29T14:03:54.352874+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:54.352874+0000","last_undegraded":"2023-03-29T14:03:54.352874+0000","last_fullsized":"2023-03-29T14:03:54.352874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773386,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":59537,"num_write":114,"num_write_kb":33984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.259185+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:03:53.259185+0000","last_peered":"2023-03-29T14:03:53.259185+0000","last_clean":"2023-03-29T14:03:53.259185+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:03:53.259185+0000","last_undegraded":"2023-03-29T14:03:53.259185+0000","last_fullsized":"2023-03-29T14:03:53.259185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":195,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.494797+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:03:53.494797+0000","last_peered":"2023-03-29T14:03:53.494797+0000","last_clean":"2023-03-29T14:03:53.494797+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:03:53.494797+0000","last_undegraded":"2023-03-29T14:03:53.494797+0000","last_fullsized":"2023-03-29T14:03:53.494797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":38109,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":165,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.258490+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:03:53.258490+0000","last_peered":"2023-03-29T14:03:53.258490+0000","last_clean":"2023-03-29T14:03:53.258490+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:03:53.258490+0000","last_undegraded":"2023-03-29T14:03:53.258490+0000","last_fullsized":"2023-03-29T14:03:53.258490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.353696+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:54.353696+0000","last_peered":"2023-03-29T14:03:54.353696+0000","last_clean":"2023-03-29T14:03:54.353696+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:54.353696+0000","last_undegraded":"2023-03-29T14:03:54.353696+0000","last_fullsized":"2023-03-29T14:03:54.353696+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":138,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.215412+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:56.215412+0000","last_peered":"2023-03-29T14:03:56.215412+0000","last_clean":"2023-03-29T14:03:56.215412+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:56.215412+0000","last_undegraded":"2023-03-29T14:03:56.215412+0000","last_fullsized":"2023-03-29T14:03:56.215412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.242602+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:03:53.242602+0000","last_peered":"2023-03-29T14:03:53.242602+0000","last_clean":"2023-03-29T14:03:53.242602+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:03:53.242602+0000","last_undegraded":"2023-03-29T14:03:53.242602+0000","last_fullsized":"2023-03-29T14:03:53.242602+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"108'36","reported_seq":168,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.572964+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:03:53.572964+0000","last_peered":"2023-03-29T14:03:53.572964+0000","last_clean":"2023-03-29T14:03:53.572964+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:03:53.572964+0000","last_undegraded":"2023-03-29T14:03:53.572964+0000","last_fullsized":"2023-03-29T14:03:53.572964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7395080,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":53,"num_read_kb":31042,"num_write":74,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-03-29T14:03:56.219419+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:56.219419+0000","last_peered":"2023-03-29T14:03:56.219419+0000","last_clean":"2023-03-29T14:03:56.219419+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:56.219419+0000","last_undegraded":"2023-03-29T14:03:56.219419+0000","last_fullsized":"2023-03-29T14:03:56.219419+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"110'53","reported_seq":210,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:55.372310+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:03:55.372310+0000","last_peered":"2023-03-29T14:03:55.372310+0000","last_clean":"2023-03-29T14:03:55.372310+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:03:55.372310+0000","last_undegraded":"2023-03-29T14:03:55.372310+0000","last_fullsized":"2023-03-29T14:03:55.372310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19933102,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":45586,"num_write":112,"num_write_kb":33194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.248519+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:03:53.248519+0000","last_peered":"2023-03-29T14:03:53.248519+0000","last_clean":"2023-03-29T14:03:53.248519+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:03:53.248519+0000","last_undegraded":"2023-03-29T14:03:53.248519+0000","last_fullsized":"2023-03-29T14:03:53.248519+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":310,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.335885+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:54.335885+0000","last_peered":"2023-03-29T14:03:54.335885+0000","last_clean":"2023-03-29T14:03:54.335885+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:54.335885+0000","last_undegraded":"2023-03-29T14:03:54.335885+0000","last_fullsized":"2023-03-29T14:03:54.335885+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":223,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.244344+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:03:53.244344+0000","last_peered":"2023-03-29T14:03:53.244344+0000","last_clean":"2023-03-29T14:03:53.244344+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:03:53.244344+0000","last_undegraded":"2023-03-29T14:03:53.244344+0000","last_fullsized":"2023-03-29T14:03:53.244344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":116,"num_read_kb":73205,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.247973+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:03:53.247973+0000","last_peered":"2023-03-29T14:03:53.247973+0000","last_clean":"2023-03-29T14:03:53.247973+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:03:53.247973+0000","last_undegraded":"2023-03-29T14:03:53.247973+0000","last_fullsized":"2023-03-29T14:03:53.247973+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":338,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.406769+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:53.406769+0000","last_peered":"2023-03-29T14:03:53.406769+0000","last_clean":"2023-03-29T14:03:53.406769+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:53.406769+0000","last_undegraded":"2023-03-29T14:03:53.406769+0000","last_fullsized":"2023-03-29T14:03:53.406769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":148,"num_read_kb":75810,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.330293+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:54.330293+0000","last_peered":"2023-03-29T14:03:54.330293+0000","last_clean":"2023-03-29T14:03:54.330293+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:54.330293+0000","last_undegraded":"2023-03-29T14:03:54.330293+0000","last_fullsized":"2023-03-29T14:03:54.330293+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":168,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.338422+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:54.338422+0000","last_peered":"2023-03-29T14:03:54.338422+0000","last_clean":"2023-03-29T14:03:54.338422+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:54.338422+0000","last_undegraded":"2023-03-29T14:03:54.338422+0000","last_fullsized":"2023-03-29T14:03:54.338422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"105'67","reported_seq":231,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:55.240129+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:55.240129+0000","last_peered":"2023-03-29T14:03:55.240129+0000","last_clean":"2023-03-29T14:03:55.240129+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:55.240129+0000","last_undegraded":"2023-03-29T14:03:55.240129+0000","last_fullsized":"2023-03-29T14:03:55.240129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479280,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":79,"num_read_kb":34993,"num_write":144,"num_write_kb":45382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-03-29T14:03:54.351474+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:54.351474+0000","last_peered":"2023-03-29T14:03:54.351474+0000","last_clean":"2023-03-29T14:03:54.351474+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:54.351474+0000","last_undegraded":"2023-03-29T14:03:54.351474+0000","last_fullsized":"2023-03-29T14:03:54.351474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.252180+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:03:53.252180+0000","last_peered":"2023-03-29T14:03:53.252180+0000","last_clean":"2023-03-29T14:03:53.252180+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:03:53.252180+0000","last_undegraded":"2023-03-29T14:03:53.252180+0000","last_fullsized":"2023-03-29T14:03:53.252180+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'51","reported_seq":210,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-03-29T14:03:53.378284+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:03:53.378284+0000","last_peered":"2023-03-29T14:03:53.378284+0000","last_clean":"2023-03-29T14:03:53.378284+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:03:53.378284+0000","last_undegraded":"2023-03-29T14:03:53.378284+0000","last_fullsized":"2023-03-29T14:03:53.378284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20571702,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":85,"num_read_kb":45127,"num_write":106,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":234390092,"num_objects":294,"num_object_clones":194,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1143,"num_read_kb":628369,"num_write":1544,"num_write_kb":449144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":156196,"kb_used_data":148828,"kb_used_omap":0,"kb_used_meta":7360,"kb_avail":94215644,"statfs":{"total":96636764160,"available":96476819456,"internally_reserved":0,"allocated":152399872,"data_stored":179162625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7536640},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":106160,"kb_used_data":99112,"kb_used_omap":0,"kb_used_meta":7040,"kb_avail":94265680,"statfs":{"total":96636764160,"available":96528056320,"internally_reserved":0,"allocated":101490688,"data_stored":115269114,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7208960},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":174176,"kb_used_data":162072,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94197664,"statfs":{"total":96636764160,"available":96458407936,"internally_reserved":0,"allocated":165961728,"data_stored":196445313,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705773,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":161424,"kb_used_data":149576,"kb_used_omap":0,"kb_used_meta":11840,"kb_avail":94210416,"statfs":{"total":96636764160,"available":96471465984,"internally_reserved":0,"allocated":153165824,"data_stored":181499162,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12124160},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:00.328 INFO:tasks.rados.rados.0.smithi017.stdout:440: done (1 left) 2023-03-29T14:04:00.329 INFO:tasks.rados.rados.0.smithi017.stdout:441: done (0 left) 2023-03-29T14:04:00.329 INFO:tasks.rados.rados.0.smithi017.stdout:442: read oid 48 snap -1 2023-03-29T14:04:00.329 INFO:tasks.rados.rados.0.smithi017.stdout:442: expect (ObjNum 216 snap 47 seq_num 216) 2023-03-29T14:04:00.329 INFO:tasks.rados.rados.0.smithi017.stdout:443: read oid 43 snap 42 2023-03-29T14:04:00.329 INFO:tasks.rados.rados.0.smithi017.stdout:443: expect deleted 2023-03-29T14:04:00.329 INFO:tasks.rados.rados.0.smithi017.stdout:444: write oid 50 current snap is 55 2023-03-29T14:04:00.329 INFO:tasks.rados.rados.0.smithi017.stdout:444: seq_num 226 ranges {598382=780511,2075401=451067,3107008=648769} 2023-03-29T14:04:00.336 INFO:tasks.rados.rados.0.smithi017.stdout:444: writing smithi017107884-50 from 598382 to 1378893 tid 1 2023-03-29T14:04:00.342 INFO:tasks.rados.rados.0.smithi017.stdout:444: writing smithi017107884-50 from 2075401 to 2526468 tid 2 2023-03-29T14:04:00.349 INFO:tasks.rados.rados.0.smithi017.stdout:444: writing smithi017107884-50 from 3107008 to 3755777 tid 3 2023-03-29T14:04:00.349 INFO:tasks.rados.rados.0.smithi017.stdout:445: read oid 1 snap -1 2023-03-29T14:04:00.350 INFO:tasks.rados.rados.0.smithi017.stdout:445: expect (ObjNum 96 snap 7 seq_num 96) 2023-03-29T14:04:00.350 INFO:tasks.rados.rados.0.smithi017.stdout:446: write oid 29 current snap is 55 2023-03-29T14:04:00.350 INFO:tasks.rados.rados.0.smithi017.stdout:446: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2023-03-29T14:04:00.355 INFO:tasks.rados.rados.0.smithi017.stdout:446: writing smithi017107884-29 from 615189 to 1070949 tid 1 2023-03-29T14:04:00.363 INFO:tasks.rados.rados.0.smithi017.stdout:446: writing smithi017107884-29 from 1733883 to 2444961 tid 2 2023-03-29T14:04:00.369 INFO:tasks.rados.rados.0.smithi017.stdout:446: writing smithi017107884-29 from 3134431 to 3594472 tid 3 2023-03-29T14:04:00.370 INFO:tasks.rados.rados.0.smithi017.stdout:447: snap_remove snap 49 2023-03-29T14:04:01.330 INFO:tasks.rados.rados.0.smithi017.stdout:447: done (5 left) 2023-03-29T14:04:01.330 INFO:tasks.rados.rados.0.smithi017.stdout:448: delete oid 12 current snap is 55 2023-03-29T14:04:01.330 INFO:tasks.rados.rados.0.smithi017.stdout:442: expect (ObjNum 216 snap 47 seq_num 216) 2023-03-29T14:04:01.524 INFO:tasks.rados.rados.0.smithi017.stdout:444: finishing write tid 1 to smithi017107884-50 2023-03-29T14:04:01.524 INFO:tasks.rados.rados.0.smithi017.stdout:445: expect (ObjNum 96 snap 7 seq_num 96) 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:444: finishing write tid 2 to smithi017107884-50 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:444: finishing write tid 3 to smithi017107884-50 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: finishing write tid 1 to smithi017107884-29 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: finishing write tid 2 to smithi017107884-29 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: finishing write tid 3 to smithi017107884-29 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: finishing write tid 4 to smithi017107884-29 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: finishing write tid 5 to smithi017107884-29 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: oid 29 updating version 0 to 40 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: oid 29 version 40 is already newer than 40 2023-03-29T14:04:01.712 INFO:tasks.rados.rados.0.smithi017.stdout:446: oid 29 version 40 is already newer than 40 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:446: oid 29 version 40 is already newer than 40 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 40 (ObjNum 227 snap 55 seq_num 227) dirty exists 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:446: left oid 29 (ObjNum 227 snap 55 seq_num 227) 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:444: finishing write tid 4 to smithi017107884-50 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:442: done (5 left) 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:443: done (4 left) 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:445: done (3 left) 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:446: done (2 left) 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:448: done (1 left) 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:449: read oid 5 snap 47 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:449: expect deleted 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:444: finishing write tid 5 to smithi017107884-50 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:444: oid 50 updating version 0 to 63 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:444: oid 50 version 63 is already newer than 63 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:444: oid 50 version 63 is already newer than 63 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:444: oid 50 version 63 is already newer than 63 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 63 (ObjNum 226 snap 55 seq_num 226) dirty exists 2023-03-29T14:04:01.713 INFO:tasks.rados.rados.0.smithi017.stdout:444: left oid 50 (ObjNum 226 snap 55 seq_num 226) 2023-03-29T14:04:01.714 INFO:tasks.rados.rados.0.smithi017.stdout:450: read oid 14 snap -1 2023-03-29T14:04:01.714 INFO:tasks.rados.rados.0.smithi017.stdout:450: expect (ObjNum 218 snap 49 seq_num 218) 2023-03-29T14:04:01.714 INFO:tasks.rados.rados.0.smithi017.stdout:444: done (2 left) 2023-03-29T14:04:01.714 INFO:tasks.rados.rados.0.smithi017.stdout:451: write (excl) oid 19 current snap is 55 2023-03-29T14:04:01.714 INFO:tasks.rados.rados.0.smithi017.stdout:451: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2023-03-29T14:04:01.719 INFO:tasks.rados.rados.0.smithi017.stdout:451: writing smithi017107884-19 from 648803 to 1371414 tid 1 2023-03-29T14:04:01.726 INFO:tasks.rados.rados.0.smithi017.stdout:451: writing smithi017107884-19 from 1967200 to 2681947 tid 2 2023-03-29T14:04:01.733 INFO:tasks.rados.rados.0.smithi017.stdout:451: writing smithi017107884-19 from 3421983 to 3848803 tid 3 2023-03-29T14:04:01.733 INFO:tasks.rados.rados.0.smithi017.stdout:452: read oid 43 snap 47 2023-03-29T14:04:01.733 INFO:tasks.rados.rados.0.smithi017.stdout:452: expect deleted 2023-03-29T14:04:01.733 INFO:tasks.rados.rados.0.smithi017.stdout:453: write (excl) oid 38 current snap is 55 2023-03-29T14:04:01.733 INFO:tasks.rados.rados.0.smithi017.stdout:453: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2023-03-29T14:04:01.741 INFO:tasks.rados.rados.0.smithi017.stdout:453: writing smithi017107884-38 from 665610 to 1463470 tid 1 2023-03-29T14:04:01.746 INFO:tasks.rados.rados.0.smithi017.stdout:453: writing smithi017107884-38 from 1909329 to 2367734 tid 2 2023-03-29T14:04:01.751 INFO:tasks.rados.rados.0.smithi017.stdout:453: writing smithi017107884-38 from 2816700 to 3226910 tid 3 2023-03-29T14:04:01.751 INFO:tasks.rados.rados.0.smithi017.stdout:454: write oid 13 current snap is 55 2023-03-29T14:04:01.751 INFO:tasks.rados.rados.0.smithi017.stdout:454: seq_num 231 ranges {682417=473109,1567811=602063,2844123=621482} 2023-03-29T14:04:01.757 INFO:tasks.rados.rados.0.smithi017.stdout:454: writing smithi017107884-13 from 682417 to 1155526 tid 1 2023-03-29T14:04:01.763 INFO:tasks.rados.rados.0.smithi017.stdout:454: writing smithi017107884-13 from 1567811 to 2169874 tid 2 2023-03-29T14:04:01.770 INFO:tasks.rados.rados.0.smithi017.stdout:454: writing smithi017107884-13 from 2844123 to 3465605 tid 3 2023-03-29T14:04:01.770 INFO:tasks.rados.rados.0.smithi017.stdout:455: snap_create 2023-03-29T14:04:01.770 INFO:tasks.rados.rados.0.smithi017.stdout:450: expect (ObjNum 218 snap 49 seq_num 218) 2023-03-29T14:04:02.066 INFO:tasks.rados.rados.0.smithi017.stdout:451: finishing write tid 1 to smithi017107884-19 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: finishing write tid 2 to smithi017107884-19 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: finishing write tid 3 to smithi017107884-19 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:453: finishing write tid 1 to smithi017107884-38 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: finishing write tid 4 to smithi017107884-19 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: finishing write tid 5 to smithi017107884-19 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: oid 19 updating version 0 to 57 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: oid 19 version 57 is already newer than 57 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: oid 19 version 57 is already newer than 57 2023-03-29T14:04:02.067 INFO:tasks.rados.rados.0.smithi017.stdout:451: oid 19 version 57 is already newer than 53 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 57 (ObjNum 229 snap 55 seq_num 229) dirty exists 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:451: left oid 19 (ObjNum 229 snap 55 seq_num 229) 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:454: finishing write tid 1 to smithi017107884-13 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:453: finishing write tid 2 to smithi017107884-38 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:454: finishing write tid 2 to smithi017107884-13 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:453: finishing write tid 3 to smithi017107884-38 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:453: finishing write tid 4 to smithi017107884-38 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:453: finishing write tid 5 to smithi017107884-38 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:453: oid 38 updating version 0 to 8 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:453: oid 38 updating version 8 to 12 2023-03-29T14:04:02.068 INFO:tasks.rados.rados.0.smithi017.stdout:453: oid 38 version 12 is already newer than 12 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:453: oid 38 version 12 is already newer than 12 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 12 (ObjNum 230 snap 55 seq_num 230) dirty exists 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:453: left oid 38 (ObjNum 230 snap 55 seq_num 230) 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:454: finishing write tid 3 to smithi017107884-13 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:454: finishing write tid 4 to smithi017107884-13 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:454: finishing write tid 5 to smithi017107884-13 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:454: oid 13 updating version 0 to 76 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:454: oid 13 version 76 is already newer than 75 2023-03-29T14:04:02.069 INFO:tasks.rados.rados.0.smithi017.stdout:454: oid 13 version 76 is already newer than 76 2023-03-29T14:04:02.070 INFO:tasks.rados.rados.0.smithi017.stdout:454: oid 13 version 76 is already newer than 76 2023-03-29T14:04:02.070 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 76 (ObjNum 231 snap 55 seq_num 231) dirty exists 2023-03-29T14:04:02.070 INFO:tasks.rados.rados.0.smithi017.stdout:454: left oid 13 (ObjNum 231 snap 55 seq_num 231) 2023-03-29T14:04:02.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:02.337 INFO:tasks.rados.rados.0.smithi017.stdout:449: done (6 left) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:450: done (5 left) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:451: done (4 left) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:452: done (3 left) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:453: done (2 left) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:454: done (1 left) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:455: done (0 left) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:456: read oid 33 snap 55 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:456: expect (ObjNum 199 snap 41 seq_num 199) 2023-03-29T14:04:02.338 INFO:tasks.rados.rados.0.smithi017.stdout:457: snap_create 2023-03-29T14:04:02.419 INFO:tasks.rados.rados.0.smithi017.stdout:456: expect (ObjNum 199 snap 41 seq_num 199) 2023-03-29T14:04:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:02.470+0000 7ff8af283700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:02.474+0000 7ff8af283700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:03.192 INFO:tasks.rados.rados.0.smithi017.stdout:456: done (1 left) 2023-03-29T14:04:03.193 INFO:tasks.rados.rados.0.smithi017.stdout:457: done (0 left) 2023-03-29T14:04:03.193 INFO:tasks.rados.rados.0.smithi017.stdout:458: write (excl) oid 45 current snap is 57 2023-03-29T14:04:03.193 INFO:tasks.rados.rados.0.smithi017.stdout:458: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2023-03-29T14:04:03.198 INFO:tasks.rados.rados.0.smithi017.stdout:458: writing smithi017107884-45 from 699224 to 1247582 tid 1 2023-03-29T14:04:03.205 INFO:tasks.rados.rados.0.smithi017.stdout:458: writing smithi017107884-45 from 2026293 to 2488367 tid 2 2023-03-29T14:04:03.212 INFO:tasks.rados.rados.0.smithi017.stdout:458: writing smithi017107884-45 from 3271546 to 3704300 tid 3 2023-03-29T14:04:03.212 INFO:tasks.rados.rados.0.smithi017.stdout:459: snap_create 2023-03-29T14:04:03.307 INFO:tasks.rados.rados.0.smithi017.stdout:458: finishing write tid 1 to smithi017107884-45 2023-03-29T14:04:03.340 INFO:tasks.rados.rados.0.smithi017.stdout:458: finishing write tid 2 to smithi017107884-45 2023-03-29T14:04:03.342 INFO:tasks.rados.rados.0.smithi017.stdout:458: finishing write tid 3 to smithi017107884-45 2023-03-29T14:04:03.343 INFO:tasks.rados.rados.0.smithi017.stdout:458: finishing write tid 4 to smithi017107884-45 2023-03-29T14:04:03.343 INFO:tasks.rados.rados.0.smithi017.stdout:458: finishing write tid 5 to smithi017107884-45 2023-03-29T14:04:03.344 INFO:tasks.rados.rados.0.smithi017.stdout:458: oid 45 updating version 0 to 61 2023-03-29T14:04:03.344 INFO:tasks.rados.rados.0.smithi017.stdout:458: oid 45 updating version 61 to 65 2023-03-29T14:04:03.344 INFO:tasks.rados.rados.0.smithi017.stdout:458: oid 45 version 65 is already newer than 65 2023-03-29T14:04:03.344 INFO:tasks.rados.rados.0.smithi017.stdout:458: oid 45 version 65 is already newer than 65 2023-03-29T14:04:03.344 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 65 (ObjNum 232 snap 57 seq_num 232) dirty exists 2023-03-29T14:04:03.344 INFO:tasks.rados.rados.0.smithi017.stdout:458: left oid 45 (ObjNum 232 snap 57 seq_num 232) 2023-03-29T14:04:04.197 INFO:tasks.rados.rados.0.smithi017.stdout:458: done (1 left) 2023-03-29T14:04:04.197 INFO:tasks.rados.rados.0.smithi017.stdout:459: done (0 left) 2023-03-29T14:04:04.197 INFO:tasks.rados.rados.0.smithi017.stdout:460: read oid 35 snap 52 2023-03-29T14:04:04.197 INFO:tasks.rados.rados.0.smithi017.stdout:460: expect deleted 2023-03-29T14:04:04.197 INFO:tasks.rados.rados.0.smithi017.stdout:461: read oid 2 snap -1 2023-03-29T14:04:04.197 INFO:tasks.rados.rados.0.smithi017.stdout:461: expect deleted 2023-03-29T14:04:04.197 INFO:tasks.rados.rados.0.smithi017.stdout:462: snap_remove snap 48 2023-03-29T14:04:04.556 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:04.688+0000 7f5992bc8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:04.693 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:04.692+0000 7f5992bc8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:05.197 INFO:tasks.rados.rados.0.smithi017.stdout:462: done (2 left) 2023-03-29T14:04:05.198 INFO:tasks.rados.rados.0.smithi017.stdout:463: read oid 9 snap -1 2023-03-29T14:04:05.198 INFO:tasks.rados.rados.0.smithi017.stdout:463: expect (ObjNum 164 snap 25 seq_num 164) 2023-03-29T14:04:05.198 INFO:tasks.rados.rados.0.smithi017.stdout:464: read oid 40 snap -1 2023-03-29T14:04:05.198 INFO:tasks.rados.rados.0.smithi017.stdout:464: expect (ObjNum 122 snap 14 seq_num 122) 2023-03-29T14:04:05.198 INFO:tasks.rados.rados.0.smithi017.stdout:465: write (excl) oid 24 current snap is 58 2023-03-29T14:04:05.198 INFO:tasks.rados.rados.0.smithi017.stdout:465: seq_num 233 ranges {716031=623607,2084775=605732,3298969=617062} 2023-03-29T14:04:05.205 INFO:tasks.rados.rados.0.smithi017.stdout:465: writing smithi017107884-24 from 716031 to 1339638 tid 1 2023-03-29T14:04:05.212 INFO:tasks.rados.rados.0.smithi017.stdout:465: writing smithi017107884-24 from 2084775 to 2690507 tid 2 2023-03-29T14:04:05.219 INFO:tasks.rados.rados.0.smithi017.stdout:465: writing smithi017107884-24 from 3298969 to 3916031 tid 3 2023-03-29T14:04:05.219 INFO:tasks.rados.rados.0.smithi017.stdout:460: done (4 left) 2023-03-29T14:04:05.219 INFO:tasks.rados.rados.0.smithi017.stdout:461: done (3 left) 2023-03-29T14:04:05.220 INFO:tasks.rados.rados.0.smithi017.stdout:466: read oid 12 snap -1 2023-03-29T14:04:05.220 INFO:tasks.rados.rados.0.smithi017.stdout:466: expect deleted 2023-03-29T14:04:05.220 INFO:tasks.rados.rados.0.smithi017.stdout:467: snap_remove snap 32 2023-03-29T14:04:05.300 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:05.300 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:05.301 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:05.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:05.432+0000 7fac784d4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:05.438+0000 7fac784d4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:05.642 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:05.654 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2023-03-29T14:04:05.194596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241083524,"num_objects":306,"num_object_clones":202,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1307,"num_read_kb":645416,"num_write":1858,"num_write_kb":467232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":618964,"kb_used_data":580212,"kb_used_omap":0,"kb_used_meta":38720,"kb_avail":376868396,"statfs":{"total":386547056640,"available":385913237504,"internally_reserved":0,"allocated":594137088,"data_stored":699128836,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":39649280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":9232,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.866265"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":148,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.354499+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:03:59.354499+0000","last_peered":"2023-03-29T14:03:59.354499+0000","last_clean":"2023-03-29T14:03:59.354499+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:03:59.354499+0000","last_undegraded":"2023-03-29T14:03:59.354499+0000","last_fullsized":"2023-03-29T14:03:59.354499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"46'7","reported_seq":105,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.272947+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:03:59.272947+0000","last_peered":"2023-03-29T14:03:59.272947+0000","last_clean":"2023-03-29T14:03:59.272947+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:03:59.272947+0000","last_undegraded":"2023-03-29T14:03:59.272947+0000","last_fullsized":"2023-03-29T14:03:59.272947+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437048,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10764,"num_write":16,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'85","reported_seq":272,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.374774+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:00.374774+0000","last_peered":"2023-03-29T14:04:00.374774+0000","last_clean":"2023-03-29T14:04:00.374774+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:00.374774+0000","last_undegraded":"2023-03-29T14:04:00.374774+0000","last_fullsized":"2023-03-29T14:04:00.374774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529734,"num_objects":28,"num_object_clones":18,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":91,"num_read_kb":49293,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":210,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.344630+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:00.344630+0000","last_peered":"2023-03-29T14:04:00.344630+0000","last_clean":"2023-03-29T14:04:00.344630+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:00.344630+0000","last_undegraded":"2023-03-29T14:04:00.344630+0000","last_fullsized":"2023-03-29T14:04:00.344630+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":77,"num_read_kb":31798,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"112'60","reported_seq":229,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.250044+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:03:59.250044+0000","last_peered":"2023-03-29T14:03:59.250044+0000","last_clean":"2023-03-29T14:03:59.250044+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:03:59.250044+0000","last_undegraded":"2023-03-29T14:03:59.250044+0000","last_fullsized":"2023-03-29T14:03:59.250044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193594,"num_objects":26,"num_object_clones":18,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":103,"num_read_kb":59538,"num_write":124,"num_write_kb":37810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":87,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.353949+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:00.353949+0000","last_peered":"2023-03-29T14:04:00.353949+0000","last_clean":"2023-03-29T14:04:00.353949+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:00.353949+0000","last_undegraded":"2023-03-29T14:04:00.353949+0000","last_fullsized":"2023-03-29T14:04:00.353949+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":199,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.339117+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:00.339117+0000","last_peered":"2023-03-29T14:04:00.339117+0000","last_clean":"2023-03-29T14:04:00.339117+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:00.339117+0000","last_undegraded":"2023-03-29T14:04:00.339117+0000","last_fullsized":"2023-03-29T14:04:00.339117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":38109,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":169,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.347804+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:00.347804+0000","last_peered":"2023-03-29T14:04:00.347804+0000","last_clean":"2023-03-29T14:04:00.347804+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:00.347804+0000","last_undegraded":"2023-03-29T14:04:00.347804+0000","last_fullsized":"2023-03-29T14:04:00.347804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.285684+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:03:59.285684+0000","last_peered":"2023-03-29T14:03:59.285684+0000","last_clean":"2023-03-29T14:03:59.285684+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:03:59.285684+0000","last_undegraded":"2023-03-29T14:03:59.285684+0000","last_fullsized":"2023-03-29T14:03:59.285684+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":141,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.357199+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:03:59.357199+0000","last_peered":"2023-03-29T14:03:59.357199+0000","last_clean":"2023-03-29T14:03:59.357199+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:03:59.357199+0000","last_undegraded":"2023-03-29T14:03:59.357199+0000","last_fullsized":"2023-03-29T14:03:59.357199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.345267+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:00.345267+0000","last_peered":"2023-03-29T14:04:00.345267+0000","last_clean":"2023-03-29T14:04:00.345267+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:00.345267+0000","last_undegraded":"2023-03-29T14:04:00.345267+0000","last_fullsized":"2023-03-29T14:04:00.345267+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"115'40","reported_seq":181,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.508565+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:00.508565+0000","last_peered":"2023-03-29T14:04:00.508565+0000","last_clean":"2023-03-29T14:04:00.508565+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:00.508565+0000","last_undegraded":"2023-03-29T14:04:00.508565+0000","last_fullsized":"2023-03-29T14:04:00.508565+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15025458,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":31043,"num_write":84,"num_write_kb":22926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.377340+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:03:59.377340+0000","last_peered":"2023-03-29T14:03:59.377340+0000","last_clean":"2023-03-29T14:03:59.377340+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:03:59.377340+0000","last_undegraded":"2023-03-29T14:03:59.377340+0000","last_fullsized":"2023-03-29T14:03:59.377340+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":231,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.472433+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:00.472433+0000","last_peered":"2023-03-29T14:04:00.472433+0000","last_clean":"2023-03-29T14:04:00.472433+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:00.472433+0000","last_undegraded":"2023-03-29T14:04:00.472433+0000","last_fullsized":"2023-03-29T14:04:00.472433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.349649+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:00.349649+0000","last_peered":"2023-03-29T14:04:00.349649+0000","last_clean":"2023-03-29T14:04:00.349649+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:00.349649+0000","last_undegraded":"2023-03-29T14:04:00.349649+0000","last_fullsized":"2023-03-29T14:04:00.349649+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":313,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.277515+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:03:59.277515+0000","last_peered":"2023-03-29T14:03:59.277515+0000","last_clean":"2023-03-29T14:03:59.277515+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:03:59.277515+0000","last_undegraded":"2023-03-29T14:03:59.277515+0000","last_fullsized":"2023-03-29T14:03:59.277515+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":227,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.350207+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:00.350207+0000","last_peered":"2023-03-29T14:04:00.350207+0000","last_clean":"2023-03-29T14:04:00.350207+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:00.350207+0000","last_undegraded":"2023-03-29T14:04:00.350207+0000","last_fullsized":"2023-03-29T14:04:00.350207+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":116,"num_read_kb":73205,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.355382+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:00.355382+0000","last_peered":"2023-03-29T14:04:00.355382+0000","last_clean":"2023-03-29T14:04:00.355382+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:00.355382+0000","last_undegraded":"2023-03-29T14:04:00.355382+0000","last_fullsized":"2023-03-29T14:04:00.355382+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":342,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.379679+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:03:59.379679+0000","last_peered":"2023-03-29T14:03:59.379679+0000","last_clean":"2023-03-29T14:03:59.379679+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:03:59.379679+0000","last_undegraded":"2023-03-29T14:03:59.379679+0000","last_fullsized":"2023-03-29T14:03:59.379679+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":148,"num_read_kb":75810,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.248887+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:03:59.248887+0000","last_peered":"2023-03-29T14:03:59.248887+0000","last_clean":"2023-03-29T14:03:59.248887+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:03:59.248887+0000","last_undegraded":"2023-03-29T14:03:59.248887+0000","last_fullsized":"2023-03-29T14:03:59.248887+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":171,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.295824+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:03:59.295824+0000","last_peered":"2023-03-29T14:03:59.295824+0000","last_clean":"2023-03-29T14:03:59.295824+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:03:59.295824+0000","last_undegraded":"2023-03-29T14:03:59.295824+0000","last_fullsized":"2023-03-29T14:03:59.295824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"114'71","reported_seq":236,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.318555+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:03:59.318555+0000","last_peered":"2023-03-29T14:03:59.318555+0000","last_clean":"2023-03-29T14:03:59.318555+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:03:59.318555+0000","last_undegraded":"2023-03-29T14:03:59.318555+0000","last_fullsized":"2023-03-29T14:03:59.318555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252270,"num_objects":32,"num_object_clones":22,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":81,"num_read_kb":34993,"num_write":144,"num_write_kb":45382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:03:59.279294+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:03:59.279294+0000","last_peered":"2023-03-29T14:03:59.279294+0000","last_clean":"2023-03-29T14:03:59.279294+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:03:59.279294+0000","last_undegraded":"2023-03-29T14:03:59.279294+0000","last_fullsized":"2023-03-29T14:03:59.279294+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.352730+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:00.352730+0000","last_peered":"2023-03-29T14:04:00.352730+0000","last_clean":"2023-03-29T14:04:00.352730+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:00.352730+0000","last_undegraded":"2023-03-29T14:04:00.352730+0000","last_fullsized":"2023-03-29T14:04:00.352730+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'51","reported_seq":217,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-03-29T14:04:00.450331+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:00.450331+0000","last_peered":"2023-03-29T14:04:00.450331+0000","last_clean":"2023-03-29T14:04:00.450331+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:00.450331+0000","last_undegraded":"2023-03-29T14:04:00.450331+0000","last_fullsized":"2023-03-29T14:04:00.450331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20571702,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":92,"num_read_kb":52818,"num_write":106,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":239902788,"num_objects":302,"num_object_clones":202,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1165,"num_read_kb":645296,"num_write":1584,"num_write_kb":463444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":765,"ondisk_log_size":765,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":158208,"kb_used_data":150776,"kb_used_omap":0,"kb_used_meta":7424,"kb_avail":94213632,"statfs":{"total":96636764160,"available":96474759168,"internally_reserved":0,"allocated":154394624,"data_stored":181616766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7602176},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":115032,"kb_used_data":107856,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94256808,"statfs":{"total":96636764160,"available":96518971392,"internally_reserved":0,"allocated":110444544,"data_stored":126601029,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":179036,"kb_used_data":166804,"kb_used_omap":0,"kb_used_meta":12224,"kb_avail":94192804,"statfs":{"total":96636764160,"available":96453431296,"internally_reserved":0,"allocated":170807296,"data_stored":202690533,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12517376},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":166688,"kb_used_data":154776,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94205152,"statfs":{"total":96636764160,"available":96466075648,"internally_reserved":0,"allocated":158490624,"data_stored":188220508,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:06.205 INFO:tasks.rados.rados.0.smithi017.stdout:467: done (4 left) 2023-03-29T14:04:06.205 INFO:tasks.rados.rados.0.smithi017.stdout:468: delete oid 47 current snap is 58 2023-03-29T14:04:06.205 INFO:tasks.rados.rados.0.smithi017.stdout:465: finishing write tid 1 to smithi017107884-24 2023-03-29T14:04:06.205 INFO:tasks.rados.rados.0.smithi017.stdout:463: expect (ObjNum 164 snap 25 seq_num 164) 2023-03-29T14:04:06.386 INFO:tasks.rados.rados.0.smithi017.stdout:465: finishing write tid 2 to smithi017107884-24 2023-03-29T14:04:06.386 INFO:tasks.rados.rados.0.smithi017.stdout:464: expect (ObjNum 122 snap 14 seq_num 122) 2023-03-29T14:04:06.504 INFO:tasks.rados.rados.0.smithi017.stdout:465: finishing write tid 3 to smithi017107884-24 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: finishing write tid 4 to smithi017107884-24 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: finishing write tid 5 to smithi017107884-24 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: oid 24 updating version 0 to 45 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: oid 24 version 45 is already newer than 45 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: oid 24 version 45 is already newer than 45 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: oid 24 version 45 is already newer than 41 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 45 (ObjNum 233 snap 58 seq_num 233) dirty exists 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: left oid 24 (ObjNum 233 snap 58 seq_num 233) 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:463: done (4 left) 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:464: done (3 left) 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:465: done (2 left) 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:466: done (1 left) 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:468: done (0 left) 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:469: read oid 15 snap 55 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:469: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:06.505 INFO:tasks.rados.rados.0.smithi017.stdout:470: snap_create 2023-03-29T14:04:06.547 INFO:tasks.rados.rados.0.smithi017.stdout:469: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:06.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:06.897 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:06.895+0000 7f5a86dfc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:06.900 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:06.899+0000 7f5a86dfc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:07.208 INFO:tasks.rados.rados.0.smithi017.stdout:469: done (1 left) 2023-03-29T14:04:07.208 INFO:tasks.rados.rados.0.smithi017.stdout:470: done (0 left) 2023-03-29T14:04:07.208 INFO:tasks.rados.rados.0.smithi017.stdout:471: delete oid 12 current snap is 59 2023-03-29T14:04:07.275 INFO:tasks.rados.rados.0.smithi017.stdout:471: done (0 left) 2023-03-29T14:04:07.275 INFO:tasks.rados.rados.0.smithi017.stdout:472: snap_remove snap 52 2023-03-29T14:04:08.211 INFO:tasks.rados.rados.0.smithi017.stdout:472: done (0 left) 2023-03-29T14:04:08.212 INFO:tasks.rados.rados.0.smithi017.stdout:473: snap_create 2023-03-29T14:04:08.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:09.122 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:09.120+0000 7f3d90054700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:09.126 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:09.125+0000 7f3d90054700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:09.213 INFO:tasks.rados.rados.0.smithi017.stdout:473: done (0 left) 2023-03-29T14:04:09.213 INFO:tasks.rados.rados.0.smithi017.stdout:474: snap_remove snap 47 2023-03-29T14:04:10.218 INFO:tasks.rados.rados.0.smithi017.stdout:474: done (0 left) 2023-03-29T14:04:10.219 INFO:tasks.rados.rados.0.smithi017.stdout:475: read oid 6 snap -1 2023-03-29T14:04:10.219 INFO:tasks.rados.rados.0.smithi017.stdout:475: expect (ObjNum 162 snap 24 seq_num 162) 2023-03-29T14:04:10.219 INFO:tasks.rados.rados.0.smithi017.stdout:476: snap_remove snap 55 2023-03-29T14:04:10.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:10.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:04:10.796 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:10.790+0000 7fe2d31c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:10.799 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:10.799+0000 7fe2d31c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:11.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:11.224 INFO:tasks.rados.rados.0.smithi017.stdout:476: done (1 left) 2023-03-29T14:04:11.224 INFO:tasks.rados.rados.0.smithi017.stdout:477: read oid 25 snap 46 2023-03-29T14:04:11.224 INFO:tasks.rados.rados.0.smithi017.stdout:477: expect (ObjNum 136 snap 17 seq_num 136) 2023-03-29T14:04:11.224 INFO:tasks.rados.rados.0.smithi017.stdout:478: delete oid 2 current snap is 60 2023-03-29T14:04:11.224 INFO:tasks.rados.rados.0.smithi017.stdout:475: expect (ObjNum 162 snap 24 seq_num 162) 2023-03-29T14:04:11.227 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:04:11.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:04:11.336 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:11.334+0000 7f7e90c1b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:11.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:11.339+0000 7f7e90c1b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:11.376 INFO:tasks.rados.rados.0.smithi017.stdout:475: done (2 left) 2023-03-29T14:04:11.376 INFO:tasks.rados.rados.0.smithi017.stdout:478: done (1 left) 2023-03-29T14:04:11.376 INFO:tasks.rados.rados.0.smithi017.stdout:479: read oid 13 snap -1 2023-03-29T14:04:11.377 INFO:tasks.rados.rados.0.smithi017.stdout:479: expect (ObjNum 231 snap 55 seq_num 231) 2023-03-29T14:04:11.377 INFO:tasks.rados.rados.0.smithi017.stdout:480: snap_create 2023-03-29T14:04:11.377 INFO:tasks.rados.rados.0.smithi017.stdout:477: expect (ObjNum 136 snap 17 seq_num 136) 2023-03-29T14:04:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:11.424+0000 7f8302aca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:11.433+0000 7f8302aca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:11.517 INFO:tasks.rados.rados.0.smithi017.stdout:479: expect (ObjNum 231 snap 55 seq_num 231) 2023-03-29T14:04:12.231 INFO:tasks.rados.rados.0.smithi017.stdout:477: done (2 left) 2023-03-29T14:04:12.232 INFO:tasks.rados.rados.0.smithi017.stdout:479: done (1 left) 2023-03-29T14:04:12.232 INFO:tasks.rados.rados.0.smithi017.stdout:480: done (0 left) 2023-03-29T14:04:12.232 INFO:tasks.rados.rados.0.smithi017.stdout:481: snap_create 2023-03-29T14:04:12.235 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:04:12.248 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:04:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:12.416+0000 7f9709f4b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:12.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:12.425+0000 7f9709f4b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:13.195 INFO:tasks.rados.rados.0.smithi017.stdout:481: done (0 left) 2023-03-29T14:04:13.195 INFO:tasks.rados.rados.0.smithi017.stdout:482: snap_remove snap 61 2023-03-29T14:04:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:04:13.210 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:04:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:13.343+0000 7f71988a4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:13.352+0000 7f71988a4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:13.427 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:13.557 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:13.555+0000 7fb801ed2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:13.560 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:13.559+0000 7fb801ed2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:14.197 INFO:tasks.rados.rados.0.smithi017.stdout:482: done (0 left) 2023-03-29T14:04:14.198 INFO:tasks.rados.rados.0.smithi017.stdout:483: delete oid 48 current snap is 62 2023-03-29T14:04:14.203 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:04:14.221 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:04:14.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:14.303 INFO:tasks.rados.rados.0.smithi017.stdout:483: done (0 left) 2023-03-29T14:04:14.303 INFO:tasks.rados.rados.0.smithi017.stdout:484: read oid 46 snap -1 2023-03-29T14:04:14.304 INFO:tasks.rados.rados.0.smithi017.stdout:484: expect deleted 2023-03-29T14:04:14.304 INFO:tasks.rados.rados.0.smithi017.stdout:485: write oid 40 current snap is 62 2023-03-29T14:04:14.304 INFO:tasks.rados.rados.0.smithi017.stdout:485: seq_num 238 ranges {400066=716205,1860832=139234} 2023-03-29T14:04:14.310 INFO:tasks.rados.rados.0.smithi017.stdout:485: writing smithi017107884-40 from 400066 to 1116271 tid 1 2023-03-29T14:04:14.315 INFO:tasks.rados.rados.0.smithi017.stdout:485: writing smithi017107884-40 from 1860832 to 2000066 tid 2 2023-03-29T14:04:14.316 INFO:tasks.rados.rados.0.smithi017.stdout:486: snap_create 2023-03-29T14:04:14.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:14.363+0000 7f362e9dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:14.370+0000 7f362e9dd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:14.438 INFO:tasks.rados.rados.0.smithi017.stdout:485: finishing write tid 1 to smithi017107884-40 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:485: finishing write tid 2 to smithi017107884-40 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:485: finishing write tid 3 to smithi017107884-40 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:485: finishing write tid 4 to smithi017107884-40 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:485: oid 40 updating version 0 to 59 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:485: oid 40 version 59 is already newer than 59 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:485: oid 40 version 59 is already newer than 59 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 59 (ObjNum 238 snap 62 seq_num 238) dirty exists 2023-03-29T14:04:14.479 INFO:tasks.rados.rados.0.smithi017.stdout:485: left oid 40 (ObjNum 238 snap 62 seq_num 238) 2023-03-29T14:04:14.575 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:14.576 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:14.588 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2023-03-29T14:04:14.195118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247470184,"num_objects":316,"num_object_clones":212,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1371,"num_read_kb":694152,"num_write":1908,"num_write_kb":483878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":890,"ondisk_log_size":890,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":648332,"kb_used_data":608940,"kb_used_omap":0,"kb_used_meta":39360,"kb_avail":376839028,"statfs":{"total":386547056640,"available":385883164672,"internally_reserved":0,"allocated":623554560,"data_stored":737838947,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":40304640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":18693,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.979277"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":154,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.437181+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:09.437181+0000","last_peered":"2023-03-29T14:04:09.437181+0000","last_clean":"2023-03-29T14:04:09.437181+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:09.437181+0000","last_undegraded":"2023-03-29T14:04:09.437181+0000","last_fullsized":"2023-03-29T14:04:09.437181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.242898+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:09.242898+0000","last_peered":"2023-03-29T14:04:09.242898+0000","last_clean":"2023-03-29T14:04:09.242898+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:09.242898+0000","last_undegraded":"2023-03-29T14:04:09.242898+0000","last_fullsized":"2023-03-29T14:04:09.242898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":10765,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'85","reported_seq":278,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.241196+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:10.241196+0000","last_peered":"2023-03-29T14:04:10.241196+0000","last_clean":"2023-03-29T14:04:10.241196+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:10.241196+0000","last_undegraded":"2023-03-29T14:04:10.241196+0000","last_fullsized":"2023-03-29T14:04:10.241196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529734,"num_objects":28,"num_object_clones":18,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":91,"num_read_kb":49293,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":225,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:11.269933+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:11.269933+0000","last_peered":"2023-03-29T14:04:11.269933+0000","last_clean":"2023-03-29T14:04:11.269933+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:11.269933+0000","last_undegraded":"2023-03-29T14:04:11.269933+0000","last_fullsized":"2023-03-29T14:04:11.269933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":49865,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"118'65","reported_seq":248,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.242492+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:10.242492+0000","last_peered":"2023-03-29T14:04:10.242492+0000","last_clean":"2023-03-29T14:04:10.242492+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:10.242492+0000","last_undegraded":"2023-03-29T14:04:10.242492+0000","last_fullsized":"2023-03-29T14:04:10.242492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496120,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":106,"num_read_kb":59539,"num_write":134,"num_write_kb":40632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.242965+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:10.242965+0000","last_peered":"2023-03-29T14:04:10.242965+0000","last_clean":"2023-03-29T14:04:10.242965+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:10.242965+0000","last_undegraded":"2023-03-29T14:04:10.242965+0000","last_fullsized":"2023-03-29T14:04:10.242965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":208,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.239367+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:10.239367+0000","last_peered":"2023-03-29T14:04:10.239367+0000","last_clean":"2023-03-29T14:04:10.239367+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:10.239367+0000","last_undegraded":"2023-03-29T14:04:10.239367+0000","last_fullsized":"2023-03-29T14:04:10.239367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":47166,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":175,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.238273+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:10.238273+0000","last_peered":"2023-03-29T14:04:10.238273+0000","last_clean":"2023-03-29T14:04:10.238273+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:10.238273+0000","last_undegraded":"2023-03-29T14:04:10.238273+0000","last_fullsized":"2023-03-29T14:04:10.238273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.240102+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:10.240102+0000","last_peered":"2023-03-29T14:04:10.240102+0000","last_clean":"2023-03-29T14:04:10.240102+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:10.240102+0000","last_undegraded":"2023-03-29T14:04:10.240102+0000","last_fullsized":"2023-03-29T14:04:10.240102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":147,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.478392+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:09.478392+0000","last_peered":"2023-03-29T14:04:09.478392+0000","last_clean":"2023-03-29T14:04:09.478392+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:09.478392+0000","last_undegraded":"2023-03-29T14:04:09.478392+0000","last_fullsized":"2023-03-29T14:04:09.478392+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.240631+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:10.240631+0000","last_peered":"2023-03-29T14:04:10.240631+0000","last_clean":"2023-03-29T14:04:10.240631+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:10.240631+0000","last_undegraded":"2023-03-29T14:04:10.240631+0000","last_fullsized":"2023-03-29T14:04:10.240631+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"120'45","reported_seq":200,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.379563+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:10.379563+0000","last_peered":"2023-03-29T14:04:10.379563+0000","last_clean":"2023-03-29T14:04:10.379563+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:10.379563+0000","last_undegraded":"2023-03-29T14:04:10.379563+0000","last_fullsized":"2023-03-29T14:04:10.379563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462440,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":31044,"num_write":94,"num_write_kb":26534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.442281+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:09.442281+0000","last_peered":"2023-03-29T14:04:09.442281+0000","last_clean":"2023-03-29T14:04:09.442281+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:09.442281+0000","last_undegraded":"2023-03-29T14:04:09.442281+0000","last_fullsized":"2023-03-29T14:04:09.442281+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":241,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.232906+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:10.232906+0000","last_peered":"2023-03-29T14:04:10.232906+0000","last_clean":"2023-03-29T14:04:10.232906+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:10.232906+0000","last_undegraded":"2023-03-29T14:04:10.232906+0000","last_fullsized":"2023-03-29T14:04:10.232906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.237109+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:10.237109+0000","last_peered":"2023-03-29T14:04:10.237109+0000","last_clean":"2023-03-29T14:04:10.237109+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:10.237109+0000","last_undegraded":"2023-03-29T14:04:10.237109+0000","last_fullsized":"2023-03-29T14:04:10.237109+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":323,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.243192+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:10.243192+0000","last_peered":"2023-03-29T14:04:10.243192+0000","last_clean":"2023-03-29T14:04:10.243192+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:10.243192+0000","last_undegraded":"2023-03-29T14:04:10.243192+0000","last_fullsized":"2023-03-29T14:04:10.243192+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":236,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.259707+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:10.259707+0000","last_peered":"2023-03-29T14:04:10.259707+0000","last_clean":"2023-03-29T14:04:10.259707+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:10.259707+0000","last_undegraded":"2023-03-29T14:04:10.259707+0000","last_fullsized":"2023-03-29T14:04:10.259707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":123,"num_read_kb":80118,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.238827+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:10.238827+0000","last_peered":"2023-03-29T14:04:10.238827+0000","last_clean":"2023-03-29T14:04:10.238827+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:10.238827+0000","last_undegraded":"2023-03-29T14:04:10.238827+0000","last_fullsized":"2023-03-29T14:04:10.238827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":355,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.370441+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:10.370441+0000","last_peered":"2023-03-29T14:04:10.370441+0000","last_clean":"2023-03-29T14:04:10.370441+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:10.370441+0000","last_undegraded":"2023-03-29T14:04:10.370441+0000","last_fullsized":"2023-03-29T14:04:10.370441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":155,"num_read_kb":81985,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.250354+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:10.250354+0000","last_peered":"2023-03-29T14:04:10.250354+0000","last_clean":"2023-03-29T14:04:10.250354+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:10.250354+0000","last_undegraded":"2023-03-29T14:04:10.250354+0000","last_fullsized":"2023-03-29T14:04:10.250354+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":181,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.229293+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:10.229293+0000","last_peered":"2023-03-29T14:04:10.229293+0000","last_clean":"2023-03-29T14:04:10.229293+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:10.229293+0000","last_undegraded":"2023-03-29T14:04:10.229293+0000","last_fullsized":"2023-03-29T14:04:10.229293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"121'78","reported_seq":259,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.252368+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:10.252368+0000","last_peered":"2023-03-29T14:04:10.252368+0000","last_clean":"2023-03-29T14:04:10.252368+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:10.252368+0000","last_undegraded":"2023-03-29T14:04:10.252368+0000","last_fullsized":"2023-03-29T14:04:10.252368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15563216,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":84,"num_read_kb":34994,"num_write":154,"num_write_kb":48698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.250795+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:09.250795+0000","last_peered":"2023-03-29T14:04:09.250795+0000","last_clean":"2023-03-29T14:04:09.250795+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:09.250795+0000","last_undegraded":"2023-03-29T14:04:09.250795+0000","last_fullsized":"2023-03-29T14:04:09.250795+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.243751+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:10.243751+0000","last_peered":"2023-03-29T14:04:10.243751+0000","last_clean":"2023-03-29T14:04:10.243751+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:10.243751+0000","last_undegraded":"2023-03-29T14:04:10.243751+0000","last_fullsized":"2023-03-29T14:04:10.243751+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"116'57","reported_seq":239,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.389832+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:10.389832+0000","last_peered":"2023-03-29T14:04:10.389832+0000","last_clean":"2023-03-29T14:04:10.389832+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:10.389832+0000","last_undegraded":"2023-03-29T14:04:10.389832+0000","last_fullsized":"2023-03-29T14:04:10.389832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21613736,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":61338,"num_write":116,"num_write_kb":33922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":246289448,"num_objects":312,"num_object_clones":212,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1229,"num_read_kb":694032,"num_write":1634,"num_write_kb":480090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":161476,"kb_used_data":153916,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":94210364,"statfs":{"total":96636764160,"available":96471412736,"internally_reserved":0,"allocated":157609984,"data_stored":186080874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116952,"kb_used_data":109712,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94254888,"statfs":{"total":96636764160,"available":96517005312,"internally_reserved":0,"allocated":112345088,"data_stored":129390933,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":191092,"kb_used_data":178668,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94180748,"statfs":{"total":96636764160,"available":96441085952,"internally_reserved":0,"allocated":182956032,"data_stored":218416850,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":178812,"kb_used_data":166644,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94193028,"statfs":{"total":96636764160,"available":96453660672,"internally_reserved":0,"allocated":170643456,"data_stored":203950290,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:14.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:14.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:14.718+0000 7f6d6135f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:14.725+0000 7f6d6135f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:14.928 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:14.931 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:14.942 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2023-03-29T14:04:14.195118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247470184,"num_objects":316,"num_object_clones":212,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1371,"num_read_kb":694152,"num_write":1908,"num_write_kb":483878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":890,"ondisk_log_size":890,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":648332,"kb_used_data":608940,"kb_used_omap":0,"kb_used_meta":39360,"kb_avail":376839028,"statfs":{"total":386547056640,"available":385883164672,"internally_reserved":0,"allocated":623554560,"data_stored":737838947,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":40304640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":18693,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.979277"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":154,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.437181+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:09.437181+0000","last_peered":"2023-03-29T14:04:09.437181+0000","last_clean":"2023-03-29T14:04:09.437181+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:09.437181+0000","last_undegraded":"2023-03-29T14:04:09.437181+0000","last_fullsized":"2023-03-29T14:04:09.437181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.242898+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:09.242898+0000","last_peered":"2023-03-29T14:04:09.242898+0000","last_clean":"2023-03-29T14:04:09.242898+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:09.242898+0000","last_undegraded":"2023-03-29T14:04:09.242898+0000","last_fullsized":"2023-03-29T14:04:09.242898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":10765,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'85","reported_seq":278,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.241196+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:10.241196+0000","last_peered":"2023-03-29T14:04:10.241196+0000","last_clean":"2023-03-29T14:04:10.241196+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:10.241196+0000","last_undegraded":"2023-03-29T14:04:10.241196+0000","last_fullsized":"2023-03-29T14:04:10.241196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529734,"num_objects":28,"num_object_clones":18,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":91,"num_read_kb":49293,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":225,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:11.269933+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:11.269933+0000","last_peered":"2023-03-29T14:04:11.269933+0000","last_clean":"2023-03-29T14:04:11.269933+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:11.269933+0000","last_undegraded":"2023-03-29T14:04:11.269933+0000","last_fullsized":"2023-03-29T14:04:11.269933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":49865,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"118'65","reported_seq":248,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.242492+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:10.242492+0000","last_peered":"2023-03-29T14:04:10.242492+0000","last_clean":"2023-03-29T14:04:10.242492+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:10.242492+0000","last_undegraded":"2023-03-29T14:04:10.242492+0000","last_fullsized":"2023-03-29T14:04:10.242492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496120,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":106,"num_read_kb":59539,"num_write":134,"num_write_kb":40632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.242965+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:10.242965+0000","last_peered":"2023-03-29T14:04:10.242965+0000","last_clean":"2023-03-29T14:04:10.242965+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:10.242965+0000","last_undegraded":"2023-03-29T14:04:10.242965+0000","last_fullsized":"2023-03-29T14:04:10.242965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":208,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.239367+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:10.239367+0000","last_peered":"2023-03-29T14:04:10.239367+0000","last_clean":"2023-03-29T14:04:10.239367+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:10.239367+0000","last_undegraded":"2023-03-29T14:04:10.239367+0000","last_fullsized":"2023-03-29T14:04:10.239367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":47166,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":175,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.238273+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:10.238273+0000","last_peered":"2023-03-29T14:04:10.238273+0000","last_clean":"2023-03-29T14:04:10.238273+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:10.238273+0000","last_undegraded":"2023-03-29T14:04:10.238273+0000","last_fullsized":"2023-03-29T14:04:10.238273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.240102+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:10.240102+0000","last_peered":"2023-03-29T14:04:10.240102+0000","last_clean":"2023-03-29T14:04:10.240102+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:10.240102+0000","last_undegraded":"2023-03-29T14:04:10.240102+0000","last_fullsized":"2023-03-29T14:04:10.240102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":147,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.478392+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:09.478392+0000","last_peered":"2023-03-29T14:04:09.478392+0000","last_clean":"2023-03-29T14:04:09.478392+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:09.478392+0000","last_undegraded":"2023-03-29T14:04:09.478392+0000","last_fullsized":"2023-03-29T14:04:09.478392+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.240631+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:10.240631+0000","last_peered":"2023-03-29T14:04:10.240631+0000","last_clean":"2023-03-29T14:04:10.240631+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:10.240631+0000","last_undegraded":"2023-03-29T14:04:10.240631+0000","last_fullsized":"2023-03-29T14:04:10.240631+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"120'45","reported_seq":200,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.379563+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:10.379563+0000","last_peered":"2023-03-29T14:04:10.379563+0000","last_clean":"2023-03-29T14:04:10.379563+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:10.379563+0000","last_undegraded":"2023-03-29T14:04:10.379563+0000","last_fullsized":"2023-03-29T14:04:10.379563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462440,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":31044,"num_write":94,"num_write_kb":26534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.442281+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:09.442281+0000","last_peered":"2023-03-29T14:04:09.442281+0000","last_clean":"2023-03-29T14:04:09.442281+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:09.442281+0000","last_undegraded":"2023-03-29T14:04:09.442281+0000","last_fullsized":"2023-03-29T14:04:09.442281+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":241,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.232906+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:10.232906+0000","last_peered":"2023-03-29T14:04:10.232906+0000","last_clean":"2023-03-29T14:04:10.232906+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:10.232906+0000","last_undegraded":"2023-03-29T14:04:10.232906+0000","last_fullsized":"2023-03-29T14:04:10.232906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.237109+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:10.237109+0000","last_peered":"2023-03-29T14:04:10.237109+0000","last_clean":"2023-03-29T14:04:10.237109+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:10.237109+0000","last_undegraded":"2023-03-29T14:04:10.237109+0000","last_fullsized":"2023-03-29T14:04:10.237109+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":323,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.243192+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:10.243192+0000","last_peered":"2023-03-29T14:04:10.243192+0000","last_clean":"2023-03-29T14:04:10.243192+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:10.243192+0000","last_undegraded":"2023-03-29T14:04:10.243192+0000","last_fullsized":"2023-03-29T14:04:10.243192+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":236,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.259707+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:10.259707+0000","last_peered":"2023-03-29T14:04:10.259707+0000","last_clean":"2023-03-29T14:04:10.259707+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:10.259707+0000","last_undegraded":"2023-03-29T14:04:10.259707+0000","last_fullsized":"2023-03-29T14:04:10.259707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":123,"num_read_kb":80118,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.238827+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:10.238827+0000","last_peered":"2023-03-29T14:04:10.238827+0000","last_clean":"2023-03-29T14:04:10.238827+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:10.238827+0000","last_undegraded":"2023-03-29T14:04:10.238827+0000","last_fullsized":"2023-03-29T14:04:10.238827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":355,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.370441+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:10.370441+0000","last_peered":"2023-03-29T14:04:10.370441+0000","last_clean":"2023-03-29T14:04:10.370441+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:10.370441+0000","last_undegraded":"2023-03-29T14:04:10.370441+0000","last_fullsized":"2023-03-29T14:04:10.370441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":155,"num_read_kb":81985,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.250354+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:10.250354+0000","last_peered":"2023-03-29T14:04:10.250354+0000","last_clean":"2023-03-29T14:04:10.250354+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:10.250354+0000","last_undegraded":"2023-03-29T14:04:10.250354+0000","last_fullsized":"2023-03-29T14:04:10.250354+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":181,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.229293+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:10.229293+0000","last_peered":"2023-03-29T14:04:10.229293+0000","last_clean":"2023-03-29T14:04:10.229293+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:10.229293+0000","last_undegraded":"2023-03-29T14:04:10.229293+0000","last_fullsized":"2023-03-29T14:04:10.229293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"121'78","reported_seq":259,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.252368+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:10.252368+0000","last_peered":"2023-03-29T14:04:10.252368+0000","last_clean":"2023-03-29T14:04:10.252368+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:10.252368+0000","last_undegraded":"2023-03-29T14:04:10.252368+0000","last_fullsized":"2023-03-29T14:04:10.252368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15563216,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":84,"num_read_kb":34994,"num_write":154,"num_write_kb":48698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.250795+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:09.250795+0000","last_peered":"2023-03-29T14:04:09.250795+0000","last_clean":"2023-03-29T14:04:09.250795+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:09.250795+0000","last_undegraded":"2023-03-29T14:04:09.250795+0000","last_fullsized":"2023-03-29T14:04:09.250795+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.243751+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:10.243751+0000","last_peered":"2023-03-29T14:04:10.243751+0000","last_clean":"2023-03-29T14:04:10.243751+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:10.243751+0000","last_undegraded":"2023-03-29T14:04:10.243751+0000","last_fullsized":"2023-03-29T14:04:10.243751+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"116'57","reported_seq":239,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.389832+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:10.389832+0000","last_peered":"2023-03-29T14:04:10.389832+0000","last_clean":"2023-03-29T14:04:10.389832+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:10.389832+0000","last_undegraded":"2023-03-29T14:04:10.389832+0000","last_fullsized":"2023-03-29T14:04:10.389832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21613736,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":61338,"num_write":116,"num_write_kb":33922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":246289448,"num_objects":312,"num_object_clones":212,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1229,"num_read_kb":694032,"num_write":1634,"num_write_kb":480090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":161476,"kb_used_data":153916,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":94210364,"statfs":{"total":96636764160,"available":96471412736,"internally_reserved":0,"allocated":157609984,"data_stored":186080874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116952,"kb_used_data":109712,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94254888,"statfs":{"total":96636764160,"available":96517005312,"internally_reserved":0,"allocated":112345088,"data_stored":129390933,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":191092,"kb_used_data":178668,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94180748,"statfs":{"total":96636764160,"available":96441085952,"internally_reserved":0,"allocated":182956032,"data_stored":218416850,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":178812,"kb_used_data":166644,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94193028,"statfs":{"total":96636764160,"available":96453660672,"internally_reserved":0,"allocated":170643456,"data_stored":203950290,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:14.948 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:04:14.948 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:14.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:15.076+0000 7fe50ecca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:15.083+0000 7fe50ecca700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:15.239 INFO:tasks.rados.rados.0.smithi017.stdout:484: done (2 left) 2023-03-29T14:04:15.239 INFO:tasks.rados.rados.0.smithi017.stdout:485: done (1 left) 2023-03-29T14:04:15.239 INFO:tasks.rados.rados.0.smithi017.stdout:486: done (0 left) 2023-03-29T14:04:15.239 INFO:tasks.rados.rados.0.smithi017.stdout:487: snap_create 2023-03-29T14:04:15.289 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:15.291 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:15.303 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2023-03-29T14:04:15.238076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247470184,"num_objects":316,"num_object_clones":212,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1371,"num_read_kb":694152,"num_write":1908,"num_write_kb":483878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":890,"ondisk_log_size":890,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":648332,"kb_used_data":608940,"kb_used_omap":0,"kb_used_meta":39360,"kb_avail":376839028,"statfs":{"total":386547056640,"available":385883164672,"internally_reserved":0,"allocated":623554560,"data_stored":737838947,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":40304640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":18693,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016774"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":154,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.437181+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:09.437181+0000","last_peered":"2023-03-29T14:04:09.437181+0000","last_clean":"2023-03-29T14:04:09.437181+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:09.437181+0000","last_undegraded":"2023-03-29T14:04:09.437181+0000","last_fullsized":"2023-03-29T14:04:09.437181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.242898+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:09.242898+0000","last_peered":"2023-03-29T14:04:09.242898+0000","last_clean":"2023-03-29T14:04:09.242898+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:09.242898+0000","last_undegraded":"2023-03-29T14:04:09.242898+0000","last_fullsized":"2023-03-29T14:04:09.242898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":10765,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'85","reported_seq":278,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.241196+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:10.241196+0000","last_peered":"2023-03-29T14:04:10.241196+0000","last_clean":"2023-03-29T14:04:10.241196+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:10.241196+0000","last_undegraded":"2023-03-29T14:04:10.241196+0000","last_fullsized":"2023-03-29T14:04:10.241196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529734,"num_objects":28,"num_object_clones":18,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":91,"num_read_kb":49293,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"66'54","reported_seq":225,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:11.269933+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:11.269933+0000","last_peered":"2023-03-29T14:04:11.269933+0000","last_clean":"2023-03-29T14:04:11.269933+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:11.269933+0000","last_undegraded":"2023-03-29T14:04:11.269933+0000","last_fullsized":"2023-03-29T14:04:11.269933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185108,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":49865,"num_write":112,"num_write_kb":32180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"118'65","reported_seq":248,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.242492+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:10.242492+0000","last_peered":"2023-03-29T14:04:10.242492+0000","last_clean":"2023-03-29T14:04:10.242492+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:10.242492+0000","last_undegraded":"2023-03-29T14:04:10.242492+0000","last_fullsized":"2023-03-29T14:04:10.242492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496120,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":106,"num_read_kb":59539,"num_write":134,"num_write_kb":40632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":93,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.242965+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:10.242965+0000","last_peered":"2023-03-29T14:04:10.242965+0000","last_clean":"2023-03-29T14:04:10.242965+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:10.242965+0000","last_undegraded":"2023-03-29T14:04:10.242965+0000","last_fullsized":"2023-03-29T14:04:10.242965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":208,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.239367+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:10.239367+0000","last_peered":"2023-03-29T14:04:10.239367+0000","last_clean":"2023-03-29T14:04:10.239367+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:10.239367+0000","last_undegraded":"2023-03-29T14:04:10.239367+0000","last_fullsized":"2023-03-29T14:04:10.239367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":47166,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":175,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.238273+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:10.238273+0000","last_peered":"2023-03-29T14:04:10.238273+0000","last_clean":"2023-03-29T14:04:10.238273+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:10.238273+0000","last_undegraded":"2023-03-29T14:04:10.238273+0000","last_fullsized":"2023-03-29T14:04:10.238273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.240102+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:10.240102+0000","last_peered":"2023-03-29T14:04:10.240102+0000","last_clean":"2023-03-29T14:04:10.240102+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:10.240102+0000","last_undegraded":"2023-03-29T14:04:10.240102+0000","last_fullsized":"2023-03-29T14:04:10.240102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":147,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.478392+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:09.478392+0000","last_peered":"2023-03-29T14:04:09.478392+0000","last_clean":"2023-03-29T14:04:09.478392+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:09.478392+0000","last_undegraded":"2023-03-29T14:04:09.478392+0000","last_fullsized":"2023-03-29T14:04:09.478392+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.240631+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:10.240631+0000","last_peered":"2023-03-29T14:04:10.240631+0000","last_clean":"2023-03-29T14:04:10.240631+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:10.240631+0000","last_undegraded":"2023-03-29T14:04:10.240631+0000","last_fullsized":"2023-03-29T14:04:10.240631+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"120'45","reported_seq":200,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.379563+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:10.379563+0000","last_peered":"2023-03-29T14:04:10.379563+0000","last_clean":"2023-03-29T14:04:10.379563+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:10.379563+0000","last_undegraded":"2023-03-29T14:04:10.379563+0000","last_fullsized":"2023-03-29T14:04:10.379563+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462440,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":31044,"num_write":94,"num_write_kb":26534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.442281+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:09.442281+0000","last_peered":"2023-03-29T14:04:09.442281+0000","last_clean":"2023-03-29T14:04:09.442281+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:09.442281+0000","last_undegraded":"2023-03-29T14:04:09.442281+0000","last_fullsized":"2023-03-29T14:04:09.442281+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":241,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.232906+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:10.232906+0000","last_peered":"2023-03-29T14:04:10.232906+0000","last_clean":"2023-03-29T14:04:10.232906+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:10.232906+0000","last_undegraded":"2023-03-29T14:04:10.232906+0000","last_fullsized":"2023-03-29T14:04:10.232906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.237109+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:10.237109+0000","last_peered":"2023-03-29T14:04:10.237109+0000","last_clean":"2023-03-29T14:04:10.237109+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:10.237109+0000","last_undegraded":"2023-03-29T14:04:10.237109+0000","last_fullsized":"2023-03-29T14:04:10.237109+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":323,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.243192+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:10.243192+0000","last_peered":"2023-03-29T14:04:10.243192+0000","last_clean":"2023-03-29T14:04:10.243192+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:10.243192+0000","last_undegraded":"2023-03-29T14:04:10.243192+0000","last_fullsized":"2023-03-29T14:04:10.243192+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":236,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.259707+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:10.259707+0000","last_peered":"2023-03-29T14:04:10.259707+0000","last_clean":"2023-03-29T14:04:10.259707+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:10.259707+0000","last_undegraded":"2023-03-29T14:04:10.259707+0000","last_fullsized":"2023-03-29T14:04:10.259707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":123,"num_read_kb":80118,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.238827+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:10.238827+0000","last_peered":"2023-03-29T14:04:10.238827+0000","last_clean":"2023-03-29T14:04:10.238827+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:10.238827+0000","last_undegraded":"2023-03-29T14:04:10.238827+0000","last_fullsized":"2023-03-29T14:04:10.238827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":355,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.370441+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:10.370441+0000","last_peered":"2023-03-29T14:04:10.370441+0000","last_clean":"2023-03-29T14:04:10.370441+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:10.370441+0000","last_undegraded":"2023-03-29T14:04:10.370441+0000","last_fullsized":"2023-03-29T14:04:10.370441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":155,"num_read_kb":81985,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.250354+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:10.250354+0000","last_peered":"2023-03-29T14:04:10.250354+0000","last_clean":"2023-03-29T14:04:10.250354+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:10.250354+0000","last_undegraded":"2023-03-29T14:04:10.250354+0000","last_fullsized":"2023-03-29T14:04:10.250354+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":181,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.229293+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:10.229293+0000","last_peered":"2023-03-29T14:04:10.229293+0000","last_clean":"2023-03-29T14:04:10.229293+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:10.229293+0000","last_undegraded":"2023-03-29T14:04:10.229293+0000","last_fullsized":"2023-03-29T14:04:10.229293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"121'78","reported_seq":259,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.252368+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:10.252368+0000","last_peered":"2023-03-29T14:04:10.252368+0000","last_clean":"2023-03-29T14:04:10.252368+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:10.252368+0000","last_undegraded":"2023-03-29T14:04:10.252368+0000","last_fullsized":"2023-03-29T14:04:10.252368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15563216,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":84,"num_read_kb":34994,"num_write":154,"num_write_kb":48698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:09.250795+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:09.250795+0000","last_peered":"2023-03-29T14:04:09.250795+0000","last_clean":"2023-03-29T14:04:09.250795+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:09.250795+0000","last_undegraded":"2023-03-29T14:04:09.250795+0000","last_fullsized":"2023-03-29T14:04:09.250795+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.243751+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:10.243751+0000","last_peered":"2023-03-29T14:04:10.243751+0000","last_clean":"2023-03-29T14:04:10.243751+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:10.243751+0000","last_undegraded":"2023-03-29T14:04:10.243751+0000","last_fullsized":"2023-03-29T14:04:10.243751+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"116'57","reported_seq":239,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.389832+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:10.389832+0000","last_peered":"2023-03-29T14:04:10.389832+0000","last_clean":"2023-03-29T14:04:10.389832+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:10.389832+0000","last_undegraded":"2023-03-29T14:04:10.389832+0000","last_fullsized":"2023-03-29T14:04:10.389832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21613736,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":61338,"num_write":116,"num_write_kb":33922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":246289448,"num_objects":312,"num_object_clones":212,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1229,"num_read_kb":694032,"num_write":1634,"num_write_kb":480090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":161476,"kb_used_data":153916,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":94210364,"statfs":{"total":96636764160,"available":96471412736,"internally_reserved":0,"allocated":157609984,"data_stored":186080874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116952,"kb_used_data":109712,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94254888,"statfs":{"total":96636764160,"available":96517005312,"internally_reserved":0,"allocated":112345088,"data_stored":129390933,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":191092,"kb_used_data":178668,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94180748,"statfs":{"total":96636764160,"available":96441085952,"internally_reserved":0,"allocated":182956032,"data_stored":218416850,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":178812,"kb_used_data":166644,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94193028,"statfs":{"total":96636764160,"available":96453660672,"internally_reserved":0,"allocated":170643456,"data_stored":203950290,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:15.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:15.770 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:15.769+0000 7f94caf73700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:15.773+0000 7f94caf73700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:16.242 INFO:tasks.rados.rados.0.smithi017.stdout:487: done (0 left) 2023-03-29T14:04:16.243 INFO:tasks.rados.rados.0.smithi017.stdout:488: write (excl) oid 9 current snap is 64 2023-03-29T14:04:16.243 INFO:tasks.rados.rados.0.smithi017.stdout:488: seq_num 239 ranges {416873=791454,1919314=97559} 2023-03-29T14:04:16.249 INFO:tasks.rados.rados.0.smithi017.stdout:488: writing smithi017107884-9 from 416873 to 1208327 tid 1 2023-03-29T14:04:16.253 INFO:tasks.rados.rados.0.smithi017.stdout:488: writing smithi017107884-9 from 1919314 to 2016873 tid 2 2023-03-29T14:04:16.254 INFO:tasks.rados.rados.0.smithi017.stdout:489: write oid 41 current snap is 64 2023-03-29T14:04:16.254 INFO:tasks.rados.rados.0.smithi017.stdout:489: seq_num 240 ranges {433680=466703,1577796=455884} 2023-03-29T14:04:16.258 INFO:tasks.rados.rados.0.smithi017.stdout:489: writing smithi017107884-41 from 433680 to 900383 tid 1 2023-03-29T14:04:16.265 INFO:tasks.rados.rados.0.smithi017.stdout:489: writing smithi017107884-41 from 1577796 to 2033680 tid 2 2023-03-29T14:04:16.265 INFO:tasks.rados.rados.0.smithi017.stdout:490: snap_create 2023-03-29T14:04:16.360 INFO:tasks.rados.rados.0.smithi017.stdout:488: finishing write tid 1 to smithi017107884-9 2023-03-29T14:04:16.377 INFO:tasks.rados.rados.0.smithi017.stdout:489: finishing write tid 1 to smithi017107884-41 2023-03-29T14:04:16.381 INFO:tasks.rados.rados.0.smithi017.stdout:489: finishing write tid 2 to smithi017107884-41 2023-03-29T14:04:16.385 INFO:tasks.rados.rados.0.smithi017.stdout:489: finishing write tid 3 to smithi017107884-41 2023-03-29T14:04:16.387 INFO:tasks.rados.rados.0.smithi017.stdout:489: finishing write tid 4 to smithi017107884-41 2023-03-29T14:04:16.388 INFO:tasks.rados.rados.0.smithi017.stdout:489: oid 41 updating version 0 to 83 2023-03-29T14:04:16.388 INFO:tasks.rados.rados.0.smithi017.stdout:489: oid 41 version 83 is already newer than 83 2023-03-29T14:04:16.388 INFO:tasks.rados.rados.0.smithi017.stdout:489: oid 41 version 83 is already newer than 83 2023-03-29T14:04:16.388 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 83 (ObjNum 240 snap 64 seq_num 240) dirty exists 2023-03-29T14:04:16.388 INFO:tasks.rados.rados.0.smithi017.stdout:489: left oid 41 (ObjNum 240 snap 64 seq_num 240) 2023-03-29T14:04:16.422 INFO:tasks.rados.rados.0.smithi017.stdout:488: finishing write tid 2 to smithi017107884-9 2023-03-29T14:04:16.423 INFO:tasks.rados.rados.0.smithi017.stdout:488: finishing write tid 3 to smithi017107884-9 2023-03-29T14:04:16.424 INFO:tasks.rados.rados.0.smithi017.stdout:488: finishing write tid 4 to smithi017107884-9 2023-03-29T14:04:16.425 INFO:tasks.rados.rados.0.smithi017.stdout:488: oid 9 updating version 0 to 60 2023-03-29T14:04:16.425 INFO:tasks.rados.rados.0.smithi017.stdout:488: oid 9 updating version 60 to 63 2023-03-29T14:04:16.425 INFO:tasks.rados.rados.0.smithi017.stdout:488: oid 9 version 63 is already newer than 63 2023-03-29T14:04:16.425 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 63 (ObjNum 239 snap 64 seq_num 239) dirty exists 2023-03-29T14:04:16.425 INFO:tasks.rados.rados.0.smithi017.stdout:488: left oid 9 (ObjNum 239 snap 64 seq_num 239) 2023-03-29T14:04:17.244 INFO:tasks.rados.rados.0.smithi017.stdout:488: done (2 left) 2023-03-29T14:04:17.245 INFO:tasks.rados.rados.0.smithi017.stdout:489: done (1 left) 2023-03-29T14:04:17.245 INFO:tasks.rados.rados.0.smithi017.stdout:490: done (0 left) 2023-03-29T14:04:17.245 INFO:tasks.rados.rados.0.smithi017.stdout:491: snap_remove snap 59 2023-03-29T14:04:17.846 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:17.977+0000 7f424fca6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:17.982+0000 7f424fca6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:18.250 INFO:tasks.rados.rados.0.smithi017.stdout:491: done (0 left) 2023-03-29T14:04:18.250 INFO:tasks.rados.rados.0.smithi017.stdout:492: snap_create 2023-03-29T14:04:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:492: done (0 left) 2023-03-29T14:04:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:493: read oid 32 snap -1 2023-03-29T14:04:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:493: expect (ObjNum 173 snap 29 seq_num 173) 2023-03-29T14:04:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:494: write oid 40 current snap is 66 2023-03-29T14:04:19.252 INFO:tasks.rados.rados.0.smithi017.stdout:494: seq_num 241 ranges {450487=541952,1636278=414209} 2023-03-29T14:04:19.258 INFO:tasks.rados.rados.0.smithi017.stdout:494: writing smithi017107884-40 from 450487 to 992439 tid 1 2023-03-29T14:04:19.264 INFO:tasks.rados.rados.0.smithi017.stdout:494: writing smithi017107884-40 from 1636278 to 2050487 tid 2 2023-03-29T14:04:19.264 INFO:tasks.rados.rados.0.smithi017.stdout:495: snap_create 2023-03-29T14:04:19.341 INFO:tasks.rados.rados.0.smithi017.stdout:493: expect (ObjNum 173 snap 29 seq_num 173) 2023-03-29T14:04:19.455 INFO:tasks.rados.rados.0.smithi017.stdout:494: finishing write tid 1 to smithi017107884-40 2023-03-29T14:04:19.455 INFO:tasks.rados.rados.0.smithi017.stdout:494: finishing write tid 2 to smithi017107884-40 2023-03-29T14:04:19.455 INFO:tasks.rados.rados.0.smithi017.stdout:494: finishing write tid 3 to smithi017107884-40 2023-03-29T14:04:19.455 INFO:tasks.rados.rados.0.smithi017.stdout:494: finishing write tid 4 to smithi017107884-40 2023-03-29T14:04:19.456 INFO:tasks.rados.rados.0.smithi017.stdout:494: oid 40 updating version 0 to 67 2023-03-29T14:04:19.456 INFO:tasks.rados.rados.0.smithi017.stdout:494: oid 40 version 67 is already newer than 67 2023-03-29T14:04:19.456 INFO:tasks.rados.rados.0.smithi017.stdout:494: oid 40 version 67 is already newer than 67 2023-03-29T14:04:19.456 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 67 (ObjNum 241 snap 66 seq_num 241) dirty exists 2023-03-29T14:04:19.456 INFO:tasks.rados.rados.0.smithi017.stdout:494: left oid 40 (ObjNum 241 snap 66 seq_num 241) 2023-03-29T14:04:20.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:20.186 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:20.185+0000 7fcb1e83e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:20.189 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:20.190+0000 7fcb1e83e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:20.259 INFO:tasks.rados.rados.0.smithi017.stdout:493: done (2 left) 2023-03-29T14:04:20.259 INFO:tasks.rados.rados.0.smithi017.stdout:494: done (1 left) 2023-03-29T14:04:20.260 INFO:tasks.rados.rados.0.smithi017.stdout:495: done (0 left) 2023-03-29T14:04:20.260 INFO:tasks.rados.rados.0.smithi017.stdout:496: read oid 38 snap -1 2023-03-29T14:04:20.260 INFO:tasks.rados.rados.0.smithi017.stdout:496: expect (ObjNum 230 snap 55 seq_num 230) 2023-03-29T14:04:20.260 INFO:tasks.rados.rados.0.smithi017.stdout:497: snap_remove snap 56 2023-03-29T14:04:20.307 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:20.308 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:20.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:20.438+0000 7f2b7b551700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:20.444+0000 7f2b7b551700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:20.649 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:20.651 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:20.663 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2023-03-29T14:04:20.257179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":238629834,"num_objects":322,"num_object_clones":218,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1385,"num_read_kb":701260,"num_write":1924,"num_write_kb":487288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":901,"ondisk_log_size":901,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":651012,"kb_used_data":611556,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":376836348,"statfs":{"total":386547056640,"available":385880420352,"internally_reserved":0,"allocated":626233344,"data_stored":742310073,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8840350,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":14,"num_read_kb":7108,"num_write":16,"num_write_kb":3410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015613"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":155,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.341038+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:10.341038+0000","last_peered":"2023-03-29T14:04:10.341038+0000","last_clean":"2023-03-29T14:04:10.341038+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:10.341038+0000","last_undegraded":"2023-03-29T14:04:10.341038+0000","last_fullsized":"2023-03-29T14:04:10.341038+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":126,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.384264+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:14.384264+0000","last_peered":"2023-03-29T14:04:14.384264+0000","last_clean":"2023-03-29T14:04:14.384264+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:14.384264+0000","last_undegraded":"2023-03-29T14:04:14.384264+0000","last_fullsized":"2023-03-29T14:04:14.384264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":10765,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"129'87","reported_seq":283,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.230147+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:14.230147+0000","last_peered":"2023-03-29T14:04:14.230147+0000","last_clean":"2023-03-29T14:04:14.230147+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:14.230147+0000","last_undegraded":"2023-03-29T14:04:14.230147+0000","last_fullsized":"2023-03-29T14:04:14.230147+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269110,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":93,"num_read_kb":49293,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"131'63","reported_seq":242,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:16.364400+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:16.364400+0000","last_peered":"2023-03-29T14:04:16.364400+0000","last_clean":"2023-03-29T14:04:16.364400+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:16.364400+0000","last_undegraded":"2023-03-29T14:04:16.364400+0000","last_fullsized":"2023-03-29T14:04:16.364400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12605382,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":49867,"num_write":128,"num_write_kb":35590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"118'65","reported_seq":251,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:15.259757+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:15.259757+0000","last_peered":"2023-03-29T14:04:15.259757+0000","last_clean":"2023-03-29T14:04:15.259757+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:15.259757+0000","last_undegraded":"2023-03-29T14:04:15.259757+0000","last_fullsized":"2023-03-29T14:04:15.259757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496120,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":106,"num_read_kb":59539,"num_write":134,"num_write_kb":40632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":96,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.210723+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:14.210723+0000","last_peered":"2023-03-29T14:04:14.210723+0000","last_clean":"2023-03-29T14:04:14.210723+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:14.210723+0000","last_undegraded":"2023-03-29T14:04:14.210723+0000","last_fullsized":"2023-03-29T14:04:14.210723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":211,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.224885+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:14.224885+0000","last_peered":"2023-03-29T14:04:14.224885+0000","last_clean":"2023-03-29T14:04:14.224885+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:14.224885+0000","last_undegraded":"2023-03-29T14:04:14.224885+0000","last_fullsized":"2023-03-29T14:04:14.224885+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":47166,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":178,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.220292+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:14.220292+0000","last_peered":"2023-03-29T14:04:14.220292+0000","last_clean":"2023-03-29T14:04:14.220292+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:14.220292+0000","last_undegraded":"2023-03-29T14:04:14.220292+0000","last_fullsized":"2023-03-29T14:04:14.220292+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":100,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.362304+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:14.362304+0000","last_peered":"2023-03-29T14:04:14.362304+0000","last_clean":"2023-03-29T14:04:14.362304+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:14.362304+0000","last_undegraded":"2023-03-29T14:04:14.362304+0000","last_fullsized":"2023-03-29T14:04:14.362304+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":148,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.336200+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:10.336200+0000","last_peered":"2023-03-29T14:04:10.336200+0000","last_clean":"2023-03-29T14:04:10.336200+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:10.336200+0000","last_undegraded":"2023-03-29T14:04:10.336200+0000","last_fullsized":"2023-03-29T14:04:10.336200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.207414+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:14.207414+0000","last_peered":"2023-03-29T14:04:14.207414+0000","last_clean":"2023-03-29T14:04:14.207414+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:14.207414+0000","last_undegraded":"2023-03-29T14:04:14.207414+0000","last_fullsized":"2023-03-29T14:04:14.207414+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"120'45","reported_seq":202,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-03-29T14:04:12.262304+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:12.262304+0000","last_peered":"2023-03-29T14:04:12.262304+0000","last_clean":"2023-03-29T14:04:12.262304+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:12.262304+0000","last_undegraded":"2023-03-29T14:04:12.262304+0000","last_fullsized":"2023-03-29T14:04:12.262304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462440,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":31044,"num_write":94,"num_write_kb":26534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:10.347491+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:10.347491+0000","last_peered":"2023-03-29T14:04:10.347491+0000","last_clean":"2023-03-29T14:04:10.347491+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:10.347491+0000","last_undegraded":"2023-03-29T14:04:10.347491+0000","last_fullsized":"2023-03-29T14:04:10.347491+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":244,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:15.262791+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:15.262791+0000","last_peered":"2023-03-29T14:04:15.262791+0000","last_clean":"2023-03-29T14:04:15.262791+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:15.262791+0000","last_undegraded":"2023-03-29T14:04:15.262791+0000","last_fullsized":"2023-03-29T14:04:15.262791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.220888+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:14.220888+0000","last_peered":"2023-03-29T14:04:14.220888+0000","last_clean":"2023-03-29T14:04:14.220888+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:14.220888+0000","last_undegraded":"2023-03-29T14:04:14.220888+0000","last_fullsized":"2023-03-29T14:04:14.220888+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":325,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.391311+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:14.391311+0000","last_peered":"2023-03-29T14:04:14.391311+0000","last_clean":"2023-03-29T14:04:14.391311+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:14.391311+0000","last_undegraded":"2023-03-29T14:04:14.391311+0000","last_fullsized":"2023-03-29T14:04:14.391311+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":239,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.224133+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:14.224133+0000","last_peered":"2023-03-29T14:04:14.224133+0000","last_clean":"2023-03-29T14:04:14.224133+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:14.224133+0000","last_undegraded":"2023-03-29T14:04:14.224133+0000","last_fullsized":"2023-03-29T14:04:14.224133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":123,"num_read_kb":80118,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.225545+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:14.225545+0000","last_peered":"2023-03-29T14:04:14.225545+0000","last_clean":"2023-03-29T14:04:14.225545+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:14.225545+0000","last_undegraded":"2023-03-29T14:04:14.225545+0000","last_fullsized":"2023-03-29T14:04:14.225545+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":358,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-03-29T14:04:12.259958+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:12.259958+0000","last_peered":"2023-03-29T14:04:12.259958+0000","last_clean":"2023-03-29T14:04:12.259958+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:12.259958+0000","last_undegraded":"2023-03-29T14:04:12.259958+0000","last_fullsized":"2023-03-29T14:04:12.259958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":156,"num_read_kb":81985,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":100,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.366044+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:14.366044+0000","last_peered":"2023-03-29T14:04:14.366044+0000","last_clean":"2023-03-29T14:04:14.366044+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:14.366044+0000","last_undegraded":"2023-03-29T14:04:14.366044+0000","last_fullsized":"2023-03-29T14:04:14.366044+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":183,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.394211+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:14.394211+0000","last_peered":"2023-03-29T14:04:14.394211+0000","last_clean":"2023-03-29T14:04:14.394211+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:14.394211+0000","last_undegraded":"2023-03-29T14:04:14.394211+0000","last_fullsized":"2023-03-29T14:04:14.394211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"121'78","reported_seq":265,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:15.260342+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:15.260342+0000","last_peered":"2023-03-29T14:04:15.260342+0000","last_clean":"2023-03-29T14:04:15.260342+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:15.260342+0000","last_undegraded":"2023-03-29T14:04:15.260342+0000","last_fullsized":"2023-03-29T14:04:15.260342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15563216,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":91,"num_read_kb":42100,"num_write":154,"num_write_kb":48698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.368432+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:14.368432+0000","last_peered":"2023-03-29T14:04:14.368432+0000","last_clean":"2023-03-29T14:04:14.368432+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:14.368432+0000","last_undegraded":"2023-03-29T14:04:14.368432+0000","last_fullsized":"2023-03-29T14:04:14.368432+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-03-29T14:04:14.215119+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:14.215119+0000","last_peered":"2023-03-29T14:04:14.215119+0000","last_clean":"2023-03-29T14:04:14.215119+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:14.215119+0000","last_undegraded":"2023-03-29T14:04:14.215119+0000","last_fullsized":"2023-03-29T14:04:14.215119+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"116'57","reported_seq":241,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-03-29T14:04:12.249727+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:12.249727+0000","last_peered":"2023-03-29T14:04:12.249727+0000","last_clean":"2023-03-29T14:04:12.249727+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:12.249727+0000","last_undegraded":"2023-03-29T14:04:12.249727+0000","last_fullsized":"2023-03-29T14:04:12.249727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21613736,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":61338,"num_write":116,"num_write_kb":33922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":237449098,"num_objects":318,"num_object_clones":218,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1243,"num_read_kb":701140,"num_write":1650,"num_write_kb":483500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":804,"ondisk_log_size":804,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705794,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":162340,"kb_used_data":154780,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":94209500,"statfs":{"total":96636764160,"available":96470528000,"internally_reserved":0,"allocated":158494720,"data_stored":187564302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705795,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116976,"kb_used_data":109736,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94254864,"statfs":{"total":96636764160,"available":96516980736,"internally_reserved":0,"allocated":112369664,"data_stored":129411755,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":192020,"kb_used_data":179532,"kb_used_omap":0,"kb_used_meta":12480,"kb_avail":94179820,"statfs":{"total":96636764160,"available":96440135680,"internally_reserved":0,"allocated":183840768,"data_stored":219900286,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12779520},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705793,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":179676,"kb_used_data":167508,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94192164,"statfs":{"total":96636764160,"available":96452775936,"internally_reserved":0,"allocated":171528192,"data_stored":205433730,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:21.260 INFO:tasks.rados.rados.0.smithi017.stdout:497: done (1 left) 2023-03-29T14:04:21.260 INFO:tasks.rados.rados.0.smithi017.stdout:498: write oid 9 current snap is 67 2023-03-29T14:04:21.260 INFO:tasks.rados.rados.0.smithi017.stdout:498: seq_num 242 ranges {467294=617201,1578407=488887} 2023-03-29T14:04:21.265 INFO:tasks.rados.rados.0.smithi017.stdout:498: writing smithi017107884-9 from 467294 to 1084495 tid 1 2023-03-29T14:04:21.272 INFO:tasks.rados.rados.0.smithi017.stdout:498: writing smithi017107884-9 from 1578407 to 2067294 tid 2 2023-03-29T14:04:21.272 INFO:tasks.rados.rados.0.smithi017.stdout:499: write oid 34 current snap is 67 2023-03-29T14:04:21.272 INFO:tasks.rados.rados.0.smithi017.stdout:499: seq_num 243 ranges {484101=692450,1636889=447212} 2023-03-29T14:04:21.279 INFO:tasks.rados.rados.0.smithi017.stdout:499: writing smithi017107884-34 from 484101 to 1176551 tid 1 2023-03-29T14:04:21.283 INFO:tasks.rados.rados.0.smithi017.stdout:499: writing smithi017107884-34 from 1636889 to 2084101 tid 2 2023-03-29T14:04:21.284 INFO:tasks.rados.rados.0.smithi017.stdout:500: snap_create 2023-03-29T14:04:21.284 INFO:tasks.rados.rados.0.smithi017.stdout:496: expect (ObjNum 230 snap 55 seq_num 230) 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:498: finishing write tid 1 to smithi017107884-9 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:498: finishing write tid 2 to smithi017107884-9 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:499: finishing write tid 1 to smithi017107884-34 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:499: finishing write tid 2 to smithi017107884-34 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:499: finishing write tid 3 to smithi017107884-34 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:499: finishing write tid 4 to smithi017107884-34 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:499: oid 34 updating version 0 to 69 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:499: oid 34 version 69 is already newer than 69 2023-03-29T14:04:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:499: oid 34 version 69 is already newer than 69 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 69 (ObjNum 243 snap 67 seq_num 243) dirty exists 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:499: left oid 34 (ObjNum 243 snap 67 seq_num 243) 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:498: finishing write tid 3 to smithi017107884-9 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:498: finishing write tid 4 to smithi017107884-9 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:498: oid 9 updating version 0 to 71 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:498: oid 9 version 71 is already newer than 71 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:498: oid 9 version 71 is already newer than 71 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 71 (ObjNum 242 snap 67 seq_num 242) dirty exists 2023-03-29T14:04:21.496 INFO:tasks.rados.rados.0.smithi017.stdout:498: left oid 9 (ObjNum 242 snap 67 seq_num 242) 2023-03-29T14:04:22.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:22.266 INFO:tasks.rados.rados.0.smithi017.stdout:496: done (3 left) 2023-03-29T14:04:22.267 INFO:tasks.rados.rados.0.smithi017.stdout:498: done (2 left) 2023-03-29T14:04:22.267 INFO:tasks.rados.rados.0.smithi017.stdout:499: done (1 left) 2023-03-29T14:04:22.267 INFO:tasks.rados.rados.0.smithi017.stdout:500: done (0 left) 2023-03-29T14:04:22.267 INFO:tasks.rados.rados.0.smithi017.stdout:501: write (excl) oid 44 current snap is 68 2023-03-29T14:04:22.267 INFO:tasks.rados.rados.0.smithi017.stdout:501: seq_num 244 ranges {500908=484052,1411724=689184} 2023-03-29T14:04:22.272 INFO:tasks.rados.rados.0.smithi017.stdout:501: writing smithi017107884-44 from 500908 to 984960 tid 1 2023-03-29T14:04:22.279 INFO:tasks.rados.rados.0.smithi017.stdout:501: writing smithi017107884-44 from 1411724 to 2100908 tid 2 2023-03-29T14:04:22.279 INFO:tasks.rados.rados.0.smithi017.stdout:502: write oid 49 current snap is 68 2023-03-29T14:04:22.279 INFO:tasks.rados.rados.0.smithi017.stdout:502: seq_num 245 ranges {517715=559301,1870206=247509} 2023-03-29T14:04:22.285 INFO:tasks.rados.rados.0.smithi017.stdout:502: writing smithi017107884-49 from 517715 to 1077016 tid 1 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:502: writing smithi017107884-49 from 1870206 to 2117715 tid 2 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:503: read oid 17 snap -1 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:503: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:504: read oid 34 snap -1 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:504: expect (ObjNum 243 snap 67 seq_num 243) 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:505: read oid 21 snap -1 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:505: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:04:22.293 INFO:tasks.rados.rados.0.smithi017.stdout:506: delete oid 30 current snap is 68 2023-03-29T14:04:22.401 INFO:tasks.rados.rados.0.smithi017.stdout:504: expect (ObjNum 243 snap 67 seq_num 243) 2023-03-29T14:04:22.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:22.403+0000 7ff1dd8ad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:22.409 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:22.408+0000 7ff1dd8ad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:22.507 INFO:tasks.rados.rados.0.smithi017.stdout:503: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:22.754 INFO:tasks.rados.rados.0.smithi017.stdout:505: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:04:22.849 INFO:tasks.rados.rados.0.smithi017.stdout:501: finishing write tid 1 to smithi017107884-44 2023-03-29T14:04:22.849 INFO:tasks.rados.rados.0.smithi017.stdout:502: finishing write tid 1 to smithi017107884-49 2023-03-29T14:04:22.849 INFO:tasks.rados.rados.0.smithi017.stdout:502: finishing write tid 2 to smithi017107884-49 2023-03-29T14:04:22.849 INFO:tasks.rados.rados.0.smithi017.stdout:502: finishing write tid 3 to smithi017107884-49 2023-03-29T14:04:22.849 INFO:tasks.rados.rados.0.smithi017.stdout:502: finishing write tid 4 to smithi017107884-49 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:502: oid 49 updating version 0 to 115 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:502: oid 49 version 115 is already newer than 115 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:502: oid 49 version 115 is already newer than 115 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 115 (ObjNum 245 snap 68 seq_num 245) dirty exists 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:502: left oid 49 (ObjNum 245 snap 68 seq_num 245) 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:501: finishing write tid 2 to smithi017107884-44 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:501: finishing write tid 3 to smithi017107884-44 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:501: finishing write tid 4 to smithi017107884-44 2023-03-29T14:04:22.850 INFO:tasks.rados.rados.0.smithi017.stdout:501: oid 44 updating version 0 to 38 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:501: oid 44 version 38 is already newer than 38 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:501: oid 44 version 38 is already newer than 35 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 38 (ObjNum 244 snap 68 seq_num 244) dirty exists 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:501: left oid 44 (ObjNum 244 snap 68 seq_num 244) 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:501: done (5 left) 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:502: done (4 left) 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:503: done (3 left) 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:504: done (2 left) 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:505: done (1 left) 2023-03-29T14:04:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:506: done (0 left) 2023-03-29T14:04:22.852 INFO:tasks.rados.rados.0.smithi017.stdout:507: write (excl) oid 7 current snap is 68 2023-03-29T14:04:22.852 INFO:tasks.rados.rados.0.smithi017.stdout:507: seq_num 247 ranges {551329=709799,1870817=280512} 2023-03-29T14:04:22.853 INFO:tasks.rados.rados.0.smithi017.stdout:507: writing smithi017107884-7 from 551329 to 1261128 tid 1 2023-03-29T14:04:22.859 INFO:tasks.rados.rados.0.smithi017.stdout:507: writing smithi017107884-7 from 1870817 to 2151329 tid 2 2023-03-29T14:04:22.859 INFO:tasks.rados.rados.0.smithi017.stdout:508: write (excl) oid 3 current snap is 68 2023-03-29T14:04:22.859 INFO:tasks.rados.rados.0.smithi017.stdout:508: seq_num 248 ranges {568136=785048,1929299=238837} 2023-03-29T14:04:22.866 INFO:tasks.rados.rados.0.smithi017.stdout:508: writing smithi017107884-3 from 568136 to 1353184 tid 1 2023-03-29T14:04:22.870 INFO:tasks.rados.rados.0.smithi017.stdout:508: writing smithi017107884-3 from 1929299 to 2168136 tid 2 2023-03-29T14:04:22.870 INFO:tasks.rados.rados.0.smithi017.stdout:509: snap_create 2023-03-29T14:04:22.888 INFO:tasks.rados.rados.0.smithi017.stdout:507: finishing write tid 1 to smithi017107884-7 2023-03-29T14:04:22.933 INFO:tasks.rados.rados.0.smithi017.stdout:507: finishing write tid 2 to smithi017107884-7 2023-03-29T14:04:22.933 INFO:tasks.rados.rados.0.smithi017.stdout:507: finishing write tid 3 to smithi017107884-7 2023-03-29T14:04:22.935 INFO:tasks.rados.rados.0.smithi017.stdout:507: finishing write tid 4 to smithi017107884-7 2023-03-29T14:04:22.935 INFO:tasks.rados.rados.0.smithi017.stdout:507: oid 7 updating version 0 to 39 2023-03-29T14:04:22.935 INFO:tasks.rados.rados.0.smithi017.stdout:507: oid 7 version 39 is already newer than 39 2023-03-29T14:04:22.935 INFO:tasks.rados.rados.0.smithi017.stdout:507: oid 7 version 39 is already newer than 36 2023-03-29T14:04:22.935 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 39 (ObjNum 247 snap 68 seq_num 247) dirty exists 2023-03-29T14:04:22.935 INFO:tasks.rados.rados.0.smithi017.stdout:507: left oid 7 (ObjNum 247 snap 68 seq_num 247) 2023-03-29T14:04:22.943 INFO:tasks.rados.rados.0.smithi017.stdout:508: finishing write tid 1 to smithi017107884-3 2023-03-29T14:04:22.996 INFO:tasks.rados.rados.0.smithi017.stdout:508: finishing write tid 2 to smithi017107884-3 2023-03-29T14:04:22.996 INFO:tasks.rados.rados.0.smithi017.stdout:508: finishing write tid 3 to smithi017107884-3 2023-03-29T14:04:22.997 INFO:tasks.rados.rados.0.smithi017.stdout:508: finishing write tid 4 to smithi017107884-3 2023-03-29T14:04:22.997 INFO:tasks.rados.rados.0.smithi017.stdout:508: oid 3 updating version 0 to 119 2023-03-29T14:04:22.997 INFO:tasks.rados.rados.0.smithi017.stdout:508: oid 3 version 119 is already newer than 116 2023-03-29T14:04:22.997 INFO:tasks.rados.rados.0.smithi017.stdout:508: oid 3 version 119 is already newer than 119 2023-03-29T14:04:22.997 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 119 (ObjNum 248 snap 68 seq_num 248) dirty exists 2023-03-29T14:04:22.998 INFO:tasks.rados.rados.0.smithi017.stdout:508: left oid 3 (ObjNum 248 snap 68 seq_num 248) 2023-03-29T14:04:23.194 INFO:tasks.rados.rados.0.smithi017.stdout:507: done (2 left) 2023-03-29T14:04:23.194 INFO:tasks.rados.rados.0.smithi017.stdout:508: done (1 left) 2023-03-29T14:04:23.194 INFO:tasks.rados.rados.0.smithi017.stdout:509: done (0 left) 2023-03-29T14:04:23.194 INFO:tasks.rados.rados.0.smithi017.stdout:510: write oid 18 current snap is 69 2023-03-29T14:04:23.195 INFO:tasks.rados.rados.0.smithi017.stdout:510: seq_num 249 ranges {584943=460297,1587781=597162} 2023-03-29T14:04:23.200 INFO:tasks.rados.rados.0.smithi017.stdout:510: writing smithi017107884-18 from 584943 to 1045240 tid 1 2023-03-29T14:04:23.206 INFO:tasks.rados.rados.0.smithi017.stdout:510: writing smithi017107884-18 from 1587781 to 2184943 tid 2 2023-03-29T14:04:23.206 INFO:tasks.rados.rados.0.smithi017.stdout:511: read oid 47 snap -1 2023-03-29T14:04:23.206 INFO:tasks.rados.rados.0.smithi017.stdout:511: expect deleted 2023-03-29T14:04:23.207 INFO:tasks.rados.rados.0.smithi017.stdout:512: snap_remove snap 62 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:512: done (2 left) 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:513: read oid 41 snap -1 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:513: expect (ObjNum 240 snap 64 seq_num 240) 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:514: delete oid 31 current snap is 69 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:510: finishing write tid 1 to smithi017107884-18 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:510: finishing write tid 2 to smithi017107884-18 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:510: finishing write tid 3 to smithi017107884-18 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:510: finishing write tid 4 to smithi017107884-18 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:510: oid 18 updating version 0 to 51 2023-03-29T14:04:24.199 INFO:tasks.rados.rados.0.smithi017.stdout:510: oid 18 version 51 is already newer than 51 2023-03-29T14:04:24.200 INFO:tasks.rados.rados.0.smithi017.stdout:510: oid 18 version 51 is already newer than 51 2023-03-29T14:04:24.200 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 51 (ObjNum 249 snap 69 seq_num 249) dirty exists 2023-03-29T14:04:24.200 INFO:tasks.rados.rados.0.smithi017.stdout:510: left oid 18 (ObjNum 249 snap 69 seq_num 249) 2023-03-29T14:04:24.258 INFO:tasks.rados.rados.0.smithi017.stdout:513: expect (ObjNum 240 snap 64 seq_num 240) 2023-03-29T14:04:24.378 INFO:tasks.rados.rados.0.smithi017.stdout:510: done (3 left) 2023-03-29T14:04:24.378 INFO:tasks.rados.rados.0.smithi017.stdout:511: done (2 left) 2023-03-29T14:04:24.378 INFO:tasks.rados.rados.0.smithi017.stdout:513: done (1 left) 2023-03-29T14:04:24.378 INFO:tasks.rados.rados.0.smithi017.stdout:514: done (0 left) 2023-03-29T14:04:24.378 INFO:tasks.rados.rados.0.smithi017.stdout:515: write oid 35 current snap is 69 2023-03-29T14:04:24.379 INFO:tasks.rados.rados.0.smithi017.stdout:515: seq_num 251 ranges {618557=727148,2104745=113812} 2023-03-29T14:04:24.385 INFO:tasks.rados.rados.0.smithi017.stdout:515: writing smithi017107884-35 from 618557 to 1345705 tid 1 2023-03-29T14:04:24.390 INFO:tasks.rados.rados.0.smithi017.stdout:515: writing smithi017107884-35 from 2104745 to 2218557 tid 2 2023-03-29T14:04:24.390 INFO:tasks.rados.rados.0.smithi017.stdout:516: snap_create 2023-03-29T14:04:24.401 INFO:tasks.rados.rados.0.smithi017.stdout:515: finishing write tid 1 to smithi017107884-35 2023-03-29T14:04:24.455 INFO:tasks.rados.rados.0.smithi017.stdout:515: finishing write tid 2 to smithi017107884-35 2023-03-29T14:04:24.459 INFO:tasks.rados.rados.0.smithi017.stdout:515: finishing write tid 3 to smithi017107884-35 2023-03-29T14:04:24.460 INFO:tasks.rados.rados.0.smithi017.stdout:515: finishing write tid 4 to smithi017107884-35 2023-03-29T14:04:24.460 INFO:tasks.rados.rados.0.smithi017.stdout:515: oid 35 updating version 0 to 41 2023-03-29T14:04:24.460 INFO:tasks.rados.rados.0.smithi017.stdout:515: oid 35 version 41 is already newer than 41 2023-03-29T14:04:24.460 INFO:tasks.rados.rados.0.smithi017.stdout:515: oid 35 version 41 is already newer than 39 2023-03-29T14:04:24.460 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 41 (ObjNum 251 snap 69 seq_num 251) dirty exists 2023-03-29T14:04:24.460 INFO:tasks.rados.rados.0.smithi017.stdout:515: left oid 35 (ObjNum 251 snap 69 seq_num 251) 2023-03-29T14:04:24.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:24.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:24.670+0000 7f8262efd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:24.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:24.675+0000 7f8262efd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:25.268 INFO:tasks.rados.rados.0.smithi017.stdout:515: done (1 left) 2023-03-29T14:04:25.269 INFO:tasks.rados.rados.0.smithi017.stdout:516: done (0 left) 2023-03-29T14:04:25.269 INFO:tasks.rados.rados.0.smithi017.stdout:517: snap_remove snap 60 2023-03-29T14:04:25.670 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:25.670 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:25.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:25.803 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:25.799+0000 7f01a6c4b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:25.806 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:25.806+0000 7f01a6c4b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:26.007 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:26.009 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:26.020 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2023-03-29T14:04:25.267301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235100496,"num_objects":328,"num_object_clones":224,"num_object_copies":984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1402,"num_read_kb":717007,"num_write":1948,"num_write_kb":493124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":914,"ondisk_log_size":914,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":659524,"kb_used_data":619812,"kb_used_omap":0,"kb_used_meta":39680,"kb_avail":376827836,"statfs":{"total":386547056640,"available":385871704064,"internally_reserved":0,"allocated":634687488,"data_stored":756170499,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":40632320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3529338,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":15747,"num_write":24,"num_write_kb":5836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.008759"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":158,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-03-29T14:04:17.326333+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:17.326333+0000","last_peered":"2023-03-29T14:04:17.326333+0000","last_clean":"2023-03-29T14:04:17.326333+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:17.326333+0000","last_undegraded":"2023-03-29T14:04:17.326333+0000","last_fullsized":"2023-03-29T14:04:17.326333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":133,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:20.335278+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:20.335278+0000","last_peered":"2023-03-29T14:04:20.335278+0000","last_clean":"2023-03-29T14:04:20.335278+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:20.335278+0000","last_undegraded":"2023-03-29T14:04:20.335278+0000","last_fullsized":"2023-03-29T14:04:20.335278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"129'87","reported_seq":287,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.265451+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:19.265451+0000","last_peered":"2023-03-29T14:04:19.265451+0000","last_clean":"2023-03-29T14:04:19.265451+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:19.265451+0000","last_undegraded":"2023-03-29T14:04:19.265451+0000","last_fullsized":"2023-03-29T14:04:19.265451+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269110,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":93,"num_read_kb":49293,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"136'71","reported_seq":260,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:21.323040+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:21.323040+0000","last_peered":"2023-03-29T14:04:21.323040+0000","last_clean":"2023-03-29T14:04:21.323040+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:21.323040+0000","last_undegraded":"2023-03-29T14:04:21.323040+0000","last_fullsized":"2023-03-29T14:04:21.323040+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12807066,"num_objects":32,"num_object_clones":24,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":104,"num_read_kb":49869,"num_write":144,"num_write_kb":39622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"118'65","reported_seq":254,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.271285+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:18.271285+0000","last_peered":"2023-03-29T14:04:18.271285+0000","last_clean":"2023-03-29T14:04:18.271285+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:18.271285+0000","last_undegraded":"2023-03-29T14:04:18.271285+0000","last_fullsized":"2023-03-29T14:04:18.271285+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496120,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":106,"num_read_kb":59539,"num_write":134,"num_write_kb":40632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":100,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.268734+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:19.268734+0000","last_peered":"2023-03-29T14:04:19.268734+0000","last_clean":"2023-03-29T14:04:19.268734+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:19.268734+0000","last_undegraded":"2023-03-29T14:04:19.268734+0000","last_fullsized":"2023-03-29T14:04:19.268734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"108'47","reported_seq":215,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.277159+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:19.277159+0000","last_peered":"2023-03-29T14:04:19.277159+0000","last_clean":"2023-03-29T14:04:19.277159+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:19.277159+0000","last_undegraded":"2023-03-29T14:04:19.277159+0000","last_fullsized":"2023-03-29T14:04:19.277159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18857388,"num_objects":20,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":47166,"num_write":98,"num_write_kb":27058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'35","reported_seq":182,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.264056+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:19.264056+0000","last_peered":"2023-03-29T14:04:19.264056+0000","last_clean":"2023-03-29T14:04:19.264056+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:19.264056+0000","last_undegraded":"2023-03-29T14:04:19.264056+0000","last_fullsized":"2023-03-29T14:04:19.264056+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697672,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":28360,"num_write":76,"num_write_kb":21998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":104,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.283153+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:18.283153+0000","last_peered":"2023-03-29T14:04:18.283153+0000","last_clean":"2023-03-29T14:04:18.283153+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:18.283153+0000","last_undegraded":"2023-03-29T14:04:18.283153+0000","last_fullsized":"2023-03-29T14:04:18.283153+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'31","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-03-29T14:04:17.311050+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:17.311050+0000","last_peered":"2023-03-29T14:04:17.311050+0000","last_clean":"2023-03-29T14:04:17.311050+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:17.311050+0000","last_undegraded":"2023-03-29T14:04:17.311050+0000","last_fullsized":"2023-03-29T14:04:17.311050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7126168,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":40,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.275156+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:19.275156+0000","last_peered":"2023-03-29T14:04:19.275156+0000","last_clean":"2023-03-29T14:04:19.275156+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:19.275156+0000","last_undegraded":"2023-03-29T14:04:19.275156+0000","last_fullsized":"2023-03-29T14:04:19.275156+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"120'45","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.423579+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:18.423579+0000","last_peered":"2023-03-29T14:04:18.423579+0000","last_clean":"2023-03-29T14:04:18.423579+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:18.423579+0000","last_undegraded":"2023-03-29T14:04:18.423579+0000","last_fullsized":"2023-03-29T14:04:18.423579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462440,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":31044,"num_write":94,"num_write_kb":26534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-03-29T14:04:17.309838+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:17.309838+0000","last_peered":"2023-03-29T14:04:17.309838+0000","last_clean":"2023-03-29T14:04:17.309838+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:17.309838+0000","last_undegraded":"2023-03-29T14:04:17.309838+0000","last_fullsized":"2023-03-29T14:04:17.309838+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":247,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.281658+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:18.281658+0000","last_peered":"2023-03-29T14:04:18.281658+0000","last_clean":"2023-03-29T14:04:18.281658+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:18.281658+0000","last_undegraded":"2023-03-29T14:04:18.281658+0000","last_fullsized":"2023-03-29T14:04:18.281658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.272604+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:19.272604+0000","last_peered":"2023-03-29T14:04:19.272604+0000","last_clean":"2023-03-29T14:04:19.272604+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:19.272604+0000","last_undegraded":"2023-03-29T14:04:19.272604+0000","last_fullsized":"2023-03-29T14:04:19.272604+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":329,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.271898+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:18.271898+0000","last_peered":"2023-03-29T14:04:18.271898+0000","last_clean":"2023-03-29T14:04:18.271898+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:18.271898+0000","last_undegraded":"2023-03-29T14:04:18.271898+0000","last_fullsized":"2023-03-29T14:04:18.271898+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":243,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.267397+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:19.267397+0000","last_peered":"2023-03-29T14:04:19.267397+0000","last_clean":"2023-03-29T14:04:19.267397+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:19.267397+0000","last_undegraded":"2023-03-29T14:04:19.267397+0000","last_fullsized":"2023-03-29T14:04:19.267397+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":123,"num_read_kb":80118,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.278769+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:19.278769+0000","last_peered":"2023-03-29T14:04:19.278769+0000","last_clean":"2023-03-29T14:04:19.278769+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:19.278769+0000","last_undegraded":"2023-03-29T14:04:19.278769+0000","last_fullsized":"2023-03-29T14:04:19.278769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"104'110","reported_seq":365,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.331080+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:19.331080+0000","last_peered":"2023-03-29T14:04:19.331080+0000","last_clean":"2023-03-29T14:04:19.331080+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:19.331080+0000","last_undegraded":"2023-03-29T14:04:19.331080+0000","last_fullsized":"2023-03-29T14:04:19.331080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36303120,"num_objects":36,"num_object_clones":22,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":163,"num_read_kb":90505,"num_write":226,"num_write_kb":64580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.269977+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:18.269977+0000","last_peered":"2023-03-29T14:04:18.269977+0000","last_clean":"2023-03-29T14:04:18.269977+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:18.269977+0000","last_undegraded":"2023-03-29T14:04:18.269977+0000","last_fullsized":"2023-03-29T14:04:18.269977+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"78'34","reported_seq":187,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.269338+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:18.269338+0000","last_peered":"2023-03-29T14:04:18.269338+0000","last_clean":"2023-03-29T14:04:18.269338+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:18.269338+0000","last_undegraded":"2023-03-29T14:04:18.269338+0000","last_fullsized":"2023-03-29T14:04:18.269338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7529470,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":64,"num_read_kb":44593,"num_write":74,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"131'83","reported_seq":275,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.265832+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:18.265832+0000","last_peered":"2023-03-29T14:04:18.265832+0000","last_clean":"2023-03-29T14:04:18.265832+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:18.265832+0000","last_undegraded":"2023-03-29T14:04:18.265832+0000","last_fullsized":"2023-03-29T14:04:18.265832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11832194,"num_objects":36,"num_object_clones":26,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":92,"num_read_kb":42101,"num_write":162,"num_write_kb":50502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":104,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.263943+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:18.263943+0000","last_peered":"2023-03-29T14:04:18.263943+0000","last_clean":"2023-03-29T14:04:18.263943+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:18.263943+0000","last_undegraded":"2023-03-29T14:04:18.263943+0000","last_fullsized":"2023-03-29T14:04:18.263943+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-03-29T14:04:19.264730+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:19.264730+0000","last_peered":"2023-03-29T14:04:19.264730+0000","last_clean":"2023-03-29T14:04:19.264730+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:19.264730+0000","last_undegraded":"2023-03-29T14:04:19.264730+0000","last_fullsized":"2023-03-29T14:04:19.264730+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"116'57","reported_seq":245,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-03-29T14:04:18.453132+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:18.453132+0000","last_peered":"2023-03-29T14:04:18.453132+0000","last_clean":"2023-03-29T14:04:18.453132+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:18.453132+0000","last_undegraded":"2023-03-29T14:04:18.453132+0000","last_fullsized":"2023-03-29T14:04:18.453132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21613736,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":61338,"num_write":116,"num_write_kb":33922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":233919760,"num_objects":324,"num_object_clones":224,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1260,"num_read_kb":716887,"num_write":1674,"num_write_kb":489336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":817,"ondisk_log_size":817,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":165148,"kb_used_data":157524,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94206692,"statfs":{"total":96636764160,"available":96467652608,"internally_reserved":0,"allocated":161304576,"data_stored":192177486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":116984,"kb_used_data":109744,"kb_used_omap":0,"kb_used_meta":7232,"kb_avail":94254856,"statfs":{"total":96636764160,"available":96516972544,"internally_reserved":0,"allocated":112377856,"data_stored":129418737,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7405568},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":194836,"kb_used_data":182284,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94177004,"statfs":{"total":96636764160,"available":96437252096,"internally_reserved":0,"allocated":186658816,"data_stored":224520408,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":182556,"kb_used_data":170260,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94189284,"statfs":{"total":96636764160,"available":96449826816,"internally_reserved":0,"allocated":174346240,"data_stored":210053868,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:26.272 INFO:tasks.rados.rados.0.smithi017.stdout:517: done (0 left) 2023-03-29T14:04:26.272 INFO:tasks.rados.rados.0.smithi017.stdout:518: snap_remove snap 39 2023-03-29T14:04:26.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:26.881 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:26.879+0000 7f7f83fc1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:26.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:26.884+0000 7f7f83fc1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:27.273 INFO:tasks.rados.rados.0.smithi017.stdout:518: done (0 left) 2023-03-29T14:04:27.274 INFO:tasks.rados.rados.0.smithi017.stdout:519: write (excl) oid 22 current snap is 70 2023-03-29T14:04:27.274 INFO:tasks.rados.rados.0.smithi017.stdout:519: seq_num 252 ranges {635364=402397,1763227=472137} 2023-03-29T14:04:27.279 INFO:tasks.rados.rados.0.smithi017.stdout:519: writing smithi017107884-22 from 635364 to 1037761 tid 1 2023-03-29T14:04:27.286 INFO:tasks.rados.rados.0.smithi017.stdout:519: writing smithi017107884-22 from 1763227 to 2235364 tid 2 2023-03-29T14:04:27.286 INFO:tasks.rados.rados.0.smithi017.stdout:520: read oid 23 snap 50 2023-03-29T14:04:27.286 INFO:tasks.rados.rados.0.smithi017.stdout:520: expect (ObjNum 205 snap 44 seq_num 205) 2023-03-29T14:04:27.286 INFO:tasks.rados.rados.0.smithi017.stdout:521: read oid 16 snap -1 2023-03-29T14:04:27.287 INFO:tasks.rados.rados.0.smithi017.stdout:521: expect (ObjNum 175 snap 33 seq_num 175) 2023-03-29T14:04:27.287 INFO:tasks.rados.rados.0.smithi017.stdout:522: read oid 1 snap -1 2023-03-29T14:04:27.287 INFO:tasks.rados.rados.0.smithi017.stdout:522: expect (ObjNum 96 snap 7 seq_num 96) 2023-03-29T14:04:27.287 INFO:tasks.rados.rados.0.smithi017.stdout:523: delete oid 45 current snap is 70 2023-03-29T14:04:27.386 INFO:tasks.rados.rados.0.smithi017.stdout:519: finishing write tid 1 to smithi017107884-22 2023-03-29T14:04:27.396 INFO:tasks.rados.rados.0.smithi017.stdout:522: expect (ObjNum 96 snap 7 seq_num 96) 2023-03-29T14:04:27.584 INFO:tasks.rados.rados.0.smithi017.stdout:520: expect (ObjNum 205 snap 44 seq_num 205) 2023-03-29T14:04:27.719 INFO:tasks.rados.rados.0.smithi017.stdout:519: finishing write tid 2 to smithi017107884-22 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:519: finishing write tid 3 to smithi017107884-22 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:519: finishing write tid 4 to smithi017107884-22 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:519: oid 22 updating version 0 to 29 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:519: oid 22 version 29 is already newer than 26 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:519: oid 22 version 29 is already newer than 29 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 29 (ObjNum 252 snap 70 seq_num 252) dirty exists 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:519: left oid 22 (ObjNum 252 snap 70 seq_num 252) 2023-03-29T14:04:27.720 INFO:tasks.rados.rados.0.smithi017.stdout:519: done (4 left) 2023-03-29T14:04:27.721 INFO:tasks.rados.rados.0.smithi017.stdout:520: done (3 left) 2023-03-29T14:04:27.721 INFO:tasks.rados.rados.0.smithi017.stdout:522: done (2 left) 2023-03-29T14:04:27.721 INFO:tasks.rados.rados.0.smithi017.stdout:523: done (1 left) 2023-03-29T14:04:27.721 INFO:tasks.rados.rados.0.smithi017.stdout:524: write (excl) oid 5 current snap is 70 2023-03-29T14:04:27.721 INFO:tasks.rados.rados.0.smithi017.stdout:521: expect (ObjNum 175 snap 33 seq_num 175) 2023-03-29T14:04:27.835 INFO:tasks.rados.rados.0.smithi017.stdout:524: seq_num 254 ranges {668978=552895,1763838=505140} 2023-03-29T14:04:27.842 INFO:tasks.rados.rados.0.smithi017.stdout:524: writing smithi017107884-5 from 668978 to 1221873 tid 1 2023-03-29T14:04:27.850 INFO:tasks.rados.rados.0.smithi017.stdout:524: writing smithi017107884-5 from 1763838 to 2268978 tid 2 2023-03-29T14:04:27.850 INFO:tasks.rados.rados.0.smithi017.stdout:521: done (1 left) 2023-03-29T14:04:27.850 INFO:tasks.rados.rados.0.smithi017.stdout:525: snap_remove snap 65 2023-03-29T14:04:28.279 INFO:tasks.rados.rados.0.smithi017.stdout:525: done (1 left) 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:526: read oid 40 snap 66 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:526: expect (ObjNum 241 snap 66 seq_num 241) 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:527: read oid 15 snap -1 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:527: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:528: read oid 7 snap 46 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:528: expect (ObjNum 166 snap 26 seq_num 166) 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:529: write oid 25 current snap is 70 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:524: finishing write tid 1 to smithi017107884-5 2023-03-29T14:04:28.280 INFO:tasks.rados.rados.0.smithi017.stdout:529: seq_num 255 ranges {685785=628144,1822320=463465} 2023-03-29T14:04:28.289 INFO:tasks.rados.rados.0.smithi017.stdout:529: writing smithi017107884-25 from 685785 to 1313929 tid 1 2023-03-29T14:04:28.296 INFO:tasks.rados.rados.0.smithi017.stdout:529: writing smithi017107884-25 from 1822320 to 2285785 tid 2 2023-03-29T14:04:28.296 INFO:tasks.rados.rados.0.smithi017.stdout:530: snap_remove snap 64 2023-03-29T14:04:29.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:29.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:29.139+0000 7feddf574700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:29.143 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:29.143+0000 7feddf574700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:29.280 INFO:tasks.rados.rados.0.smithi017.stdout:530: done (5 left) 2023-03-29T14:04:29.281 INFO:tasks.rados.rados.0.smithi017.stdout:531: write oid 9 current snap is 70 2023-03-29T14:04:29.281 INFO:tasks.rados.rados.0.smithi017.stdout:524: finishing write tid 2 to smithi017107884-5 2023-03-29T14:04:29.281 INFO:tasks.rados.rados.0.smithi017.stdout:531: seq_num 256 ranges {702592=703393,1880802=421790} 2023-03-29T14:04:29.288 INFO:tasks.rados.rados.0.smithi017.stdout:531: writing smithi017107884-9 from 702592 to 1405985 tid 1 2023-03-29T14:04:29.293 INFO:tasks.rados.rados.0.smithi017.stdout:531: writing smithi017107884-9 from 1880802 to 2302592 tid 2 2023-03-29T14:04:29.293 INFO:tasks.rados.rados.0.smithi017.stdout:532: read oid 47 snap 54 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:532: expect (ObjNum 192 snap 38 seq_num 192) 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:533: delete oid 46 current snap is 70 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:524: finishing write tid 3 to smithi017107884-5 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:524: finishing write tid 4 to smithi017107884-5 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:524: oid 5 updating version 0 to 86 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:524: oid 5 version 86 is already newer than 84 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:524: oid 5 version 86 is already newer than 86 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 86 (ObjNum 254 snap 70 seq_num 254) dirty exists 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:524: left oid 5 (ObjNum 254 snap 70 seq_num 254) 2023-03-29T14:04:29.294 INFO:tasks.rados.rados.0.smithi017.stdout:527: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:29.423 INFO:tasks.rados.rados.0.smithi017.stdout:526: expect (ObjNum 241 snap 66 seq_num 241) 2023-03-29T14:04:29.584 INFO:tasks.rados.rados.0.smithi017.stdout:528: expect (ObjNum 166 snap 26 seq_num 166) 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: finishing write tid 1 to smithi017107884-25 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: finishing write tid 2 to smithi017107884-25 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: finishing write tid 3 to smithi017107884-25 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: finishing write tid 4 to smithi017107884-25 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: oid 25 updating version 0 to 75 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: oid 25 version 75 is already newer than 75 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: oid 25 version 75 is already newer than 75 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 75 (ObjNum 255 snap 70 seq_num 255) dirty exists 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:529: left oid 25 (ObjNum 255 snap 70 seq_num 255) 2023-03-29T14:04:29.804 INFO:tasks.rados.rados.0.smithi017.stdout:532: expect (ObjNum 192 snap 38 seq_num 192) 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: finishing write tid 1 to smithi017107884-9 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: finishing write tid 2 to smithi017107884-9 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: finishing write tid 3 to smithi017107884-9 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: finishing write tid 4 to smithi017107884-9 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: oid 9 updating version 0 to 79 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: oid 9 version 79 is already newer than 79 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: oid 9 version 79 is already newer than 79 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 79 (ObjNum 256 snap 70 seq_num 256) dirty exists 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:531: left oid 9 (ObjNum 256 snap 70 seq_num 256) 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:524: done (7 left) 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:526: done (6 left) 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:527: done (5 left) 2023-03-29T14:04:29.968 INFO:tasks.rados.rados.0.smithi017.stdout:528: done (4 left) 2023-03-29T14:04:29.969 INFO:tasks.rados.rados.0.smithi017.stdout:529: done (3 left) 2023-03-29T14:04:29.969 INFO:tasks.rados.rados.0.smithi017.stdout:531: done (2 left) 2023-03-29T14:04:29.969 INFO:tasks.rados.rados.0.smithi017.stdout:532: done (1 left) 2023-03-29T14:04:29.969 INFO:tasks.rados.rados.0.smithi017.stdout:533: done (0 left) 2023-03-29T14:04:29.969 INFO:tasks.rados.rados.0.smithi017.stdout:534: write (excl) oid 19 current snap is 70 2023-03-29T14:04:29.969 INFO:tasks.rados.rados.0.smithi017.stdout:534: seq_num 258 ranges {736206=453891,1881413=454793} 2023-03-29T14:04:29.973 INFO:tasks.rados.rados.0.smithi017.stdout:534: writing smithi017107884-19 from 736206 to 1190097 tid 1 2023-03-29T14:04:29.980 INFO:tasks.rados.rados.0.smithi017.stdout:534: writing smithi017107884-19 from 1881413 to 2336206 tid 2 2023-03-29T14:04:29.980 INFO:tasks.rados.rados.0.smithi017.stdout:535: write (excl) oid 6 current snap is 70 2023-03-29T14:04:29.980 INFO:tasks.rados.rados.0.smithi017.stdout:535: seq_num 259 ranges {753013=645493,2056248=296765} 2023-03-29T14:04:29.987 INFO:tasks.rados.rados.0.smithi017.stdout:535: writing smithi017107884-6 from 753013 to 1398506 tid 1 2023-03-29T14:04:29.993 INFO:tasks.rados.rados.0.smithi017.stdout:535: writing smithi017107884-6 from 2056248 to 2353013 tid 2 2023-03-29T14:04:29.993 INFO:tasks.rados.rados.0.smithi017.stdout:536: delete oid 31 current snap is 70 2023-03-29T14:04:30.019 INFO:tasks.rados.rados.0.smithi017.stdout:536: done (2 left) 2023-03-29T14:04:30.019 INFO:tasks.rados.rados.0.smithi017.stdout:537: read oid 13 snap -1 2023-03-29T14:04:30.019 INFO:tasks.rados.rados.0.smithi017.stdout:537: expect (ObjNum 231 snap 55 seq_num 231) 2023-03-29T14:04:30.019 INFO:tasks.rados.rados.0.smithi017.stdout:538: write oid 5 current snap is 70 2023-03-29T14:04:30.019 INFO:tasks.rados.rados.0.smithi017.stdout:538: seq_num 261 ranges {786627=795991,2173212=213415} 2023-03-29T14:04:30.028 INFO:tasks.rados.rados.0.smithi017.stdout:538: writing smithi017107884-5 from 786627 to 1582618 tid 1 2023-03-29T14:04:30.032 INFO:tasks.rados.rados.0.smithi017.stdout:538: writing smithi017107884-5 from 2173212 to 2386627 tid 2 2023-03-29T14:04:30.032 INFO:tasks.rados.rados.0.smithi017.stdout:539: snap_remove snap 54 2023-03-29T14:04:30.287 INFO:tasks.rados.rados.0.smithi017.stdout:539: done (4 left) 2023-03-29T14:04:30.287 INFO:tasks.rados.rados.0.smithi017.stdout:540: delete oid 16 current snap is 70 2023-03-29T14:04:30.287 INFO:tasks.rados.rados.0.smithi017.stdout:535: finishing write tid 1 to smithi017107884-6 2023-03-29T14:04:30.287 INFO:tasks.rados.rados.0.smithi017.stdout:537: expect (ObjNum 231 snap 55 seq_num 231) 2023-03-29T14:04:30.441 INFO:tasks.rados.rados.0.smithi017.stdout:535: finishing write tid 2 to smithi017107884-6 2023-03-29T14:04:30.441 INFO:tasks.rados.rados.0.smithi017.stdout:534: finishing write tid 1 to smithi017107884-19 2023-03-29T14:04:30.441 INFO:tasks.rados.rados.0.smithi017.stdout:538: finishing write tid 1 to smithi017107884-5 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:538: finishing write tid 2 to smithi017107884-5 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:538: finishing write tid 3 to smithi017107884-5 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:538: finishing write tid 4 to smithi017107884-5 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:538: oid 5 updating version 0 to 89 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:538: oid 5 version 89 is already newer than 89 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:538: oid 5 version 89 is already newer than 89 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 89 (ObjNum 261 snap 70 seq_num 261) dirty exists 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:538: left oid 5 (ObjNum 261 snap 70 seq_num 261) 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:534: finishing write tid 2 to smithi017107884-19 2023-03-29T14:04:30.442 INFO:tasks.rados.rados.0.smithi017.stdout:537: done (4 left) 2023-03-29T14:04:30.443 INFO:tasks.rados.rados.0.smithi017.stdout:538: done (3 left) 2023-03-29T14:04:30.443 INFO:tasks.rados.rados.0.smithi017.stdout:540: done (2 left) 2023-03-29T14:04:30.443 INFO:tasks.rados.rados.0.smithi017.stdout:541: write (excl) oid 47 current snap is 70 2023-03-29T14:04:30.443 INFO:tasks.rados.rados.0.smithi017.stdout:534: finishing write tid 3 to smithi017107884-19 2023-03-29T14:04:30.443 INFO:tasks.rados.rados.0.smithi017.stdout:541: seq_num 263 ranges {420241=546489,1373823=544414} 2023-03-29T14:04:30.446 INFO:tasks.rados.rados.0.smithi017.stdout:541: writing smithi017107884-47 from 420241 to 966730 tid 1 2023-03-29T14:04:30.451 INFO:tasks.rados.rados.0.smithi017.stdout:541: writing smithi017107884-47 from 1373823 to 1918237 tid 2 2023-03-29T14:04:30.451 INFO:tasks.rados.rados.0.smithi017.stdout:542: write oid 2 current snap is 70 2023-03-29T14:04:30.451 INFO:tasks.rados.rados.0.smithi017.stdout:542: seq_num 264 ranges {437048=621738,1832305=404425} 2023-03-29T14:04:30.457 INFO:tasks.rados.rados.0.smithi017.stdout:542: writing smithi017107884-2 from 437048 to 1058786 tid 1 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:542: writing smithi017107884-2 from 1832305 to 2236730 tid 2 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:543: write oid 32 current snap is 70 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:534: finishing write tid 4 to smithi017107884-19 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:534: oid 19 updating version 0 to 61 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:534: oid 19 version 61 is already newer than 61 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:534: oid 19 version 61 is already newer than 58 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 61 (ObjNum 258 snap 70 seq_num 258) dirty exists 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:534: left oid 19 (ObjNum 258 snap 70 seq_num 258) 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:535: finishing write tid 3 to smithi017107884-6 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:535: finishing write tid 4 to smithi017107884-6 2023-03-29T14:04:30.464 INFO:tasks.rados.rados.0.smithi017.stdout:535: oid 6 updating version 0 to 56 2023-03-29T14:04:30.465 INFO:tasks.rados.rados.0.smithi017.stdout:535: oid 6 version 56 is already newer than 53 2023-03-29T14:04:30.465 INFO:tasks.rados.rados.0.smithi017.stdout:535: oid 6 version 56 is already newer than 56 2023-03-29T14:04:30.465 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 56 (ObjNum 259 snap 70 seq_num 259) dirty exists 2023-03-29T14:04:30.465 INFO:tasks.rados.rados.0.smithi017.stdout:535: left oid 6 (ObjNum 259 snap 70 seq_num 259) 2023-03-29T14:04:30.465 INFO:tasks.rados.rados.0.smithi017.stdout:543: seq_num 265 ranges {453855=696987,1890787=548083} 2023-03-29T14:04:30.470 INFO:tasks.rados.rados.0.smithi017.stdout:543: writing smithi017107884-32 from 453855 to 1150842 tid 1 2023-03-29T14:04:30.477 INFO:tasks.rados.rados.0.smithi017.stdout:543: writing smithi017107884-32 from 1890787 to 2438870 tid 2 2023-03-29T14:04:30.477 INFO:tasks.rados.rados.0.smithi017.stdout:534: done (4 left) 2023-03-29T14:04:30.477 INFO:tasks.rados.rados.0.smithi017.stdout:535: done (3 left) 2023-03-29T14:04:30.477 INFO:tasks.rados.rados.0.smithi017.stdout:544: delete oid 44 current snap is 70 2023-03-29T14:04:30.566 INFO:tasks.rados.rados.0.smithi017.stdout:542: finishing write tid 1 to smithi017107884-2 2023-03-29T14:04:30.567 INFO:tasks.rados.rados.0.smithi017.stdout:541: finishing write tid 1 to smithi017107884-47 2023-03-29T14:04:30.588 INFO:tasks.rados.rados.0.smithi017.stdout:542: finishing write tid 2 to smithi017107884-2 2023-03-29T14:04:30.589 INFO:tasks.rados.rados.0.smithi017.stdout:544: done (3 left) 2023-03-29T14:04:30.590 INFO:tasks.rados.rados.0.smithi017.stdout:545: write oid 7 current snap is 70 2023-03-29T14:04:30.590 INFO:tasks.rados.rados.0.smithi017.stdout:545: seq_num 267 ranges {487469=563838,1607751=551752} 2023-03-29T14:04:30.595 INFO:tasks.rados.rados.0.smithi017.stdout:545: writing smithi017107884-7 from 487469 to 1051307 tid 1 2023-03-29T14:04:30.602 INFO:tasks.rados.rados.0.smithi017.stdout:545: writing smithi017107884-7 from 1607751 to 2159503 tid 2 2023-03-29T14:04:30.602 INFO:tasks.rados.rados.0.smithi017.stdout:546: read oid 46 snap 63 2023-03-29T14:04:30.602 INFO:tasks.rados.rados.0.smithi017.stdout:546: expect deleted 2023-03-29T14:04:30.602 INFO:tasks.rados.rados.0.smithi017.stdout:547: write oid 40 current snap is 70 2023-03-29T14:04:30.602 INFO:tasks.rados.rados.0.smithi017.stdout:542: finishing write tid 3 to smithi017107884-2 2023-03-29T14:04:30.602 INFO:tasks.rados.rados.0.smithi017.stdout:547: seq_num 268 ranges {504276=639087,1666233=695410} 2023-03-29T14:04:30.608 INFO:tasks.rados.rados.0.smithi017.stdout:547: writing smithi017107884-40 from 504276 to 1143363 tid 1 2023-03-29T14:04:30.615 INFO:tasks.rados.rados.0.smithi017.stdout:547: writing smithi017107884-40 from 1666233 to 2361643 tid 2 2023-03-29T14:04:30.616 INFO:tasks.rados.rados.0.smithi017.stdout:548: snap_remove snap 66 2023-03-29T14:04:31.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:31.025 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:31.025 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:31.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:31.155+0000 7f4abeb53700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:31.162+0000 7f4abeb53700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:31.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:548: done (6 left) 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:549: write oid 11 current snap is 70 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:542: finishing write tid 4 to smithi017107884-2 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:542: oid 2 updating version 0 to 122 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:542: oid 2 version 122 is already newer than 122 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:542: oid 2 version 122 is already newer than 122 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 122 (ObjNum 264 snap 70 seq_num 264) dirty exists 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:542: left oid 2 (ObjNum 264 snap 70 seq_num 264) 2023-03-29T14:04:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:549: seq_num 269 ranges {521083=714336,1724715=555421} 2023-03-29T14:04:31.296 INFO:tasks.rados.rados.0.smithi017.stdout:549: writing smithi017107884-11 from 521083 to 1235419 tid 1 2023-03-29T14:04:31.302 INFO:tasks.rados.rados.0.smithi017.stdout:549: writing smithi017107884-11 from 1724715 to 2280136 tid 2 2023-03-29T14:04:31.303 INFO:tasks.rados.rados.0.smithi017.stdout:542: done (6 left) 2023-03-29T14:04:31.303 INFO:tasks.rados.rados.0.smithi017.stdout:550: write (excl) oid 3 current snap is 70 2023-03-29T14:04:31.303 INFO:tasks.rados.rados.0.smithi017.stdout:550: seq_num 270 ranges {537890=789585,1783197=699079} 2023-03-29T14:04:31.310 INFO:tasks.rados.rados.0.smithi017.stdout:550: writing smithi017107884-3 from 537890 to 1327475 tid 1 2023-03-29T14:04:31.316 INFO:tasks.rados.rados.0.smithi017.stdout:550: writing smithi017107884-3 from 1783197 to 2482276 tid 2 2023-03-29T14:04:31.317 INFO:tasks.rados.rados.0.smithi017.stdout:551: snap_remove snap 46 2023-03-29T14:04:31.369 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:31.371 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:31.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:31.382+0000 7f9d52c9c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:31.384 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2023-03-29T14:04:31.287479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":217958148,"num_objects":348,"num_object_clones":244,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1473,"num_read_kb":760399,"num_write":2028,"num_write_kb":513094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":957,"ondisk_log_size":957,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":694552,"kb_used_data":649396,"kb_used_omap":0,"kb_used_meta":44608,"kb_avail":376792808,"statfs":{"total":386547056640,"available":385835835392,"internally_reserved":0,"allocated":664981504,"data_stored":797023962,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":45678592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6184778,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":22260,"num_write":24,"num_write_kb":6218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015272"},"pg_stats":[{"pgid":"3.f","version":"93'25","reported_seq":162,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.490091+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:25.490091+0000","last_peered":"2023-03-29T14:04:25.490091+0000","last_clean":"2023-03-29T14:04:25.490091+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:25.490091+0000","last_undegraded":"2023-03-29T14:04:25.490091+0000","last_fullsized":"2023-03-29T14:04:25.490091+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403566,"num_objects":10,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":45184,"num_write":52,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":138,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:24.228620+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:24.228620+0000","last_peered":"2023-03-29T14:04:24.228620+0000","last_clean":"2023-03-29T14:04:24.228620+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:24.228620+0000","last_undegraded":"2023-03-29T14:04:24.228620+0000","last_fullsized":"2023-03-29T14:04:24.228620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"129'87","reported_seq":295,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.466904+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:25.466904+0000","last_peered":"2023-03-29T14:04:25.466904+0000","last_clean":"2023-03-29T14:04:25.466904+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:25.466904+0000","last_undegraded":"2023-03-29T14:04:25.466904+0000","last_fullsized":"2023-03-29T14:04:25.466904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269110,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":100,"num_read_kb":56304,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"136'71","reported_seq":265,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.475946+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:25.475946+0000","last_peered":"2023-03-29T14:04:25.475946+0000","last_clean":"2023-03-29T14:04:25.475946+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:25.475946+0000","last_undegraded":"2023-03-29T14:04:25.475946+0000","last_fullsized":"2023-03-29T14:04:25.475946+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12807066,"num_objects":32,"num_object_clones":24,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":104,"num_read_kb":49869,"num_write":144,"num_write_kb":39622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'69","reported_seq":269,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:24.231267+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:24.231267+0000","last_peered":"2023-03-29T14:04:24.231267+0000","last_clean":"2023-03-29T14:04:24.231267+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:24.231267+0000","last_undegraded":"2023-03-29T14:04:24.231267+0000","last_fullsized":"2023-03-29T14:04:24.231267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17109592,"num_objects":30,"num_object_clones":22,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":114,"num_read_kb":64813,"num_write":142,"num_write_kb":42860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":105,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.472375+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:25.472375+0000","last_peered":"2023-03-29T14:04:25.472375+0000","last_clean":"2023-03-29T14:04:25.472375+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:25.472375+0000","last_undegraded":"2023-03-29T14:04:25.472375+0000","last_fullsized":"2023-03-29T14:04:25.472375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"139'53","reported_seq":229,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.492049+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:25.492049+0000","last_peered":"2023-03-29T14:04:25.492049+0000","last_clean":"2023-03-29T14:04:25.492049+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:25.492049+0000","last_undegraded":"2023-03-29T14:04:25.492049+0000","last_fullsized":"2023-03-29T14:04:25.492049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697738,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":86,"num_read_kb":47167,"num_write":106,"num_write_kb":29126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"137'39","reported_seq":194,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.461020+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:25.461020+0000","last_peered":"2023-03-29T14:04:25.461020+0000","last_clean":"2023-03-29T14:04:25.461020+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:25.461020+0000","last_undegraded":"2023-03-29T14:04:25.461020+0000","last_fullsized":"2023-03-29T14:04:25.461020+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420406,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":28361,"num_write":84,"num_write_kb":23934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:24.210551+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:24.210551+0000","last_peered":"2023-03-29T14:04:24.210551+0000","last_clean":"2023-03-29T14:04:24.210551+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:24.210551+0000","last_undegraded":"2023-03-29T14:04:24.210551+0000","last_fullsized":"2023-03-29T14:04:24.210551+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'33","reported_seq":157,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.461256+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:25.461256+0000","last_peered":"2023-03-29T14:04:25.461256+0000","last_clean":"2023-03-29T14:04:25.461256+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:25.461256+0000","last_undegraded":"2023-03-29T14:04:25.461256+0000","last_fullsized":"2023-03-29T14:04:25.461256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":42,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":108,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.485772+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:25.485772+0000","last_peered":"2023-03-29T14:04:25.485772+0000","last_clean":"2023-03-29T14:04:25.485772+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:25.485772+0000","last_undegraded":"2023-03-29T14:04:25.485772+0000","last_fullsized":"2023-03-29T14:04:25.485772+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"120'45","reported_seq":214,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.014284+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:30.014284+0000","last_peered":"2023-03-29T14:04:30.014284+0000","last_clean":"2023-03-29T14:04:30.014284+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:30.014284+0000","last_undegraded":"2023-03-29T14:04:30.014284+0000","last_fullsized":"2023-03-29T14:04:30.014284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462440,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":31044,"num_write":94,"num_write_kb":26534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.498641+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:25.498641+0000","last_peered":"2023-03-29T14:04:25.498641+0000","last_clean":"2023-03-29T14:04:25.498641+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:25.498641+0000","last_undegraded":"2023-03-29T14:04:25.498641+0000","last_fullsized":"2023-03-29T14:04:25.498641+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":252,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:24.227990+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:24.227990+0000","last_peered":"2023-03-29T14:04:24.227990+0000","last_clean":"2023-03-29T14:04:24.227990+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:24.227990+0000","last_undegraded":"2023-03-29T14:04:24.227990+0000","last_fullsized":"2023-03-29T14:04:24.227990+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":108,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.493729+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:25.493729+0000","last_peered":"2023-03-29T14:04:25.493729+0000","last_clean":"2023-03-29T14:04:25.493729+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:25.493729+0000","last_undegraded":"2023-03-29T14:04:25.493729+0000","last_fullsized":"2023-03-29T14:04:25.493729+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":335,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.483138+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:25.483138+0000","last_peered":"2023-03-29T14:04:25.483138+0000","last_clean":"2023-03-29T14:04:25.483138+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:25.483138+0000","last_undegraded":"2023-03-29T14:04:25.483138+0000","last_fullsized":"2023-03-29T14:04:25.483138+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"85'52","reported_seq":251,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.489549+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:25.489549+0000","last_peered":"2023-03-29T14:04:25.489549+0000","last_clean":"2023-03-29T14:04:25.489549+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:25.489549+0000","last_undegraded":"2023-03-29T14:04:25.489549+0000","last_fullsized":"2023-03-29T14:04:25.489549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16168268,"num_objects":20,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":130,"num_read_kb":84068,"num_write":108,"num_write_kb":31512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.486928+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:25.486928+0000","last_peered":"2023-03-29T14:04:25.486928+0000","last_clean":"2023-03-29T14:04:25.486928+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:25.486928+0000","last_undegraded":"2023-03-29T14:04:25.486928+0000","last_fullsized":"2023-03-29T14:04:25.486928+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"145'126","reported_seq":407,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.620005+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:30.620005+0000","last_peered":"2023-03-29T14:04:30.620005+0000","last_clean":"2023-03-29T14:04:30.620005+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:30.620005+0000","last_undegraded":"2023-03-29T14:04:30.620005+0000","last_fullsized":"2023-03-29T14:04:30.620005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026118,"num_objects":42,"num_object_clones":28,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":183,"num_read_kb":106523,"num_write":258,"num_write_kb":72600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":109,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:24.227245+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:24.227245+0000","last_peered":"2023-03-29T14:04:24.227245+0000","last_clean":"2023-03-29T14:04:24.227245+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:24.227245+0000","last_undegraded":"2023-03-29T14:04:24.227245+0000","last_fullsized":"2023-03-29T14:04:24.227245+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"139'41","reported_seq":206,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:24.460980+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:24.460980+0000","last_peered":"2023-03-29T14:04:24.460980+0000","last_clean":"2023-03-29T14:04:24.460980+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:24.460980+0000","last_undegraded":"2023-03-29T14:04:24.460980+0000","last_fullsized":"2023-03-29T14:04:24.460980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8638930,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":44595,"num_write":90,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"131'83","reported_seq":283,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:24.257505+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:24.257505+0000","last_peered":"2023-03-29T14:04:24.257505+0000","last_clean":"2023-03-29T14:04:24.257505+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:24.257505+0000","last_undegraded":"2023-03-29T14:04:24.257505+0000","last_fullsized":"2023-03-29T14:04:24.257505+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11832194,"num_objects":36,"num_object_clones":26,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":99,"num_read_kb":46992,"num_write":162,"num_write_kb":50502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.481366+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:25.481366+0000","last_peered":"2023-03-29T14:04:25.481366+0000","last_clean":"2023-03-29T14:04:25.481366+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:25.481366+0000","last_undegraded":"2023-03-29T14:04:25.481366+0000","last_fullsized":"2023-03-29T14:04:25.481366+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":108,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-03-29T14:04:25.462753+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:25.462753+0000","last_peered":"2023-03-29T14:04:25.462753+0000","last_clean":"2023-03-29T14:04:25.462753+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:25.462753+0000","last_undegraded":"2023-03-29T14:04:25.462753+0000","last_fullsized":"2023-03-29T14:04:25.462753+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"144'61","reported_seq":263,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.153176+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:30.153176+0000","last_peered":"2023-03-29T14:04:30.153176+0000","last_clean":"2023-03-29T14:04:30.153176+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:30.153176+0000","last_undegraded":"2023-03-29T14:04:30.153176+0000","last_fullsized":"2023-03-29T14:04:30.153176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18588542,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":112,"num_read_kb":67582,"num_write":124,"num_write_kb":35700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":216777412,"num_objects":344,"num_object_clones":244,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1331,"num_read_kb":760279,"num_write":1754,"num_write_kb":509306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":177532,"kb_used_data":165556,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94194308,"statfs":{"total":96636764160,"available":96454971392,"internally_reserved":0,"allocated":169529344,"data_stored":203176987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119992,"kb_used_data":112624,"kb_used_omap":0,"kb_used_meta":7360,"kb_avail":94251848,"statfs":{"total":96636764160,"available":96513892352,"internally_reserved":0,"allocated":115326976,"data_stored":133673402,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7536640},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705808,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":208124,"kb_used_data":194736,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94163716,"statfs":{"total":96636764160,"available":96423645184,"internally_reserved":0,"allocated":199409664,"data_stored":241943966,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":188904,"kb_used_data":176480,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94182936,"statfs":{"total":96636764160,"available":96443326464,"internally_reserved":0,"allocated":180715520,"data_stored":218229607,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:31.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:31.387+0000 7f9d52c9c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:32.297 INFO:tasks.rados.rados.0.smithi017.stdout:551: done (7 left) 2023-03-29T14:04:32.297 INFO:tasks.rados.rados.0.smithi017.stdout:552: write (excl) oid 39 current snap is 70 2023-03-29T14:04:32.297 INFO:tasks.rados.rados.0.smithi017.stdout:552: seq_num 271 ranges {554697=464834,1725326=559090} 2023-03-29T14:04:32.302 INFO:tasks.rados.rados.0.smithi017.stdout:552: writing smithi017107884-39 from 554697 to 1019531 tid 1 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:552: writing smithi017107884-39 from 1725326 to 2284416 tid 2 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:553: read oid 29 snap 58 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:553: expect (ObjNum 227 snap 55 seq_num 227) 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:554: read oid 22 snap -1 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:554: expect (ObjNum 252 snap 70 seq_num 252) 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:555: delete oid 13 current snap is 70 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:541: finishing write tid 2 to smithi017107884-47 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:541: finishing write tid 3 to smithi017107884-47 2023-03-29T14:04:32.310 INFO:tasks.rados.rados.0.smithi017.stdout:541: finishing write tid 4 to smithi017107884-47 2023-03-29T14:04:32.311 INFO:tasks.rados.rados.0.smithi017.stdout:541: oid 47 updating version 0 to 90 2023-03-29T14:04:32.311 INFO:tasks.rados.rados.0.smithi017.stdout:541: oid 47 updating version 90 to 92 2023-03-29T14:04:32.311 INFO:tasks.rados.rados.0.smithi017.stdout:541: oid 47 version 92 is already newer than 92 2023-03-29T14:04:32.311 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 92 (ObjNum 263 snap 70 seq_num 263) dirty exists 2023-03-29T14:04:32.311 INFO:tasks.rados.rados.0.smithi017.stdout:541: left oid 47 (ObjNum 263 snap 70 seq_num 263) 2023-03-29T14:04:32.311 INFO:tasks.rados.rados.0.smithi017.stdout:543: finishing write tid 1 to smithi017107884-32 2023-03-29T14:04:32.311 INFO:tasks.rados.rados.0.smithi017.stdout:543: finishing write tid 2 to smithi017107884-32 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: finishing write tid 1 to smithi017107884-7 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: finishing write tid 2 to smithi017107884-7 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:543: finishing write tid 3 to smithi017107884-32 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:543: finishing write tid 4 to smithi017107884-32 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:543: oid 32 updating version 0 to 126 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:543: oid 32 version 126 is already newer than 126 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:543: oid 32 version 126 is already newer than 126 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 126 (ObjNum 265 snap 70 seq_num 265) dirty exists 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:543: left oid 32 (ObjNum 265 snap 70 seq_num 265) 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: finishing write tid 3 to smithi017107884-7 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: finishing write tid 4 to smithi017107884-7 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: oid 7 updating version 0 to 43 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: oid 7 version 43 is already newer than 43 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: oid 7 version 43 is already newer than 43 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 43 (ObjNum 267 snap 70 seq_num 267) dirty exists 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:545: left oid 7 (ObjNum 267 snap 70 seq_num 267) 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:547: finishing write tid 1 to smithi017107884-40 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:547: finishing write tid 2 to smithi017107884-40 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:547: finishing write tid 3 to smithi017107884-40 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:547: finishing write tid 4 to smithi017107884-40 2023-03-29T14:04:32.312 INFO:tasks.rados.rados.0.smithi017.stdout:547: oid 40 updating version 0 to 83 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:547: oid 40 version 83 is already newer than 83 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:547: oid 40 version 83 is already newer than 83 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 83 (ObjNum 268 snap 70 seq_num 268) dirty exists 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:547: left oid 40 (ObjNum 268 snap 70 seq_num 268) 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: finishing write tid 1 to smithi017107884-11 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: finishing write tid 2 to smithi017107884-11 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: finishing write tid 3 to smithi017107884-11 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: finishing write tid 4 to smithi017107884-11 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: oid 11 updating version 0 to 130 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: oid 11 version 130 is already newer than 130 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: oid 11 version 130 is already newer than 130 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 11 v 130 (ObjNum 269 snap 70 seq_num 269) dirty exists 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:549: left oid 11 (ObjNum 269 snap 70 seq_num 269) 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:550: finishing write tid 1 to smithi017107884-3 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:550: finishing write tid 2 to smithi017107884-3 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:550: finishing write tid 3 to smithi017107884-3 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:550: finishing write tid 4 to smithi017107884-3 2023-03-29T14:04:32.313 INFO:tasks.rados.rados.0.smithi017.stdout:550: oid 3 updating version 0 to 131 2023-03-29T14:04:32.314 INFO:tasks.rados.rados.0.smithi017.stdout:550: oid 3 updating version 131 to 134 2023-03-29T14:04:32.314 INFO:tasks.rados.rados.0.smithi017.stdout:550: oid 3 version 134 is already newer than 134 2023-03-29T14:04:32.314 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 134 (ObjNum 270 snap 70 seq_num 270) dirty exists 2023-03-29T14:04:32.314 INFO:tasks.rados.rados.0.smithi017.stdout:550: left oid 3 (ObjNum 270 snap 70 seq_num 270) 2023-03-29T14:04:32.386 INFO:tasks.rados.rados.0.smithi017.stdout:554: expect (ObjNum 252 snap 70 seq_num 252) 2023-03-29T14:04:32.492 INFO:tasks.rados.rados.0.smithi017.stdout:553: expect (ObjNum 227 snap 55 seq_num 227) 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: finishing write tid 1 to smithi017107884-39 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: finishing write tid 2 to smithi017107884-39 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: finishing write tid 3 to smithi017107884-39 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: finishing write tid 4 to smithi017107884-39 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: oid 39 updating version 0 to 88 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: oid 39 updating version 88 to 90 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: oid 39 version 90 is already newer than 90 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 90 (ObjNum 271 snap 70 seq_num 271) dirty exists 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:552: left oid 39 (ObjNum 271 snap 70 seq_num 271) 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:541: done (10 left) 2023-03-29T14:04:32.641 INFO:tasks.rados.rados.0.smithi017.stdout:543: done (9 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:545: done (8 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:546: done (7 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:547: done (6 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:549: done (5 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:550: done (4 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:552: done (3 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:553: done (2 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:554: done (1 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:555: done (0 left) 2023-03-29T14:04:32.642 INFO:tasks.rados.rados.0.smithi017.stdout:556: snap_remove snap 58 2023-03-29T14:04:33.196 INFO:tasks.rados.rados.0.smithi017.stdout:556: done (0 left) 2023-03-29T14:04:33.196 INFO:tasks.rados.rados.0.smithi017.stdout:557: read oid 22 snap -1 2023-03-29T14:04:33.196 INFO:tasks.rados.rados.0.smithi017.stdout:557: expect (ObjNum 252 snap 70 seq_num 252) 2023-03-29T14:04:33.196 INFO:tasks.rados.rados.0.smithi017.stdout:558: write oid 7 current snap is 70 2023-03-29T14:04:33.196 INFO:tasks.rados.rados.0.smithi017.stdout:558: seq_num 273 ranges {588311=615332,1842290=562759} 2023-03-29T14:04:33.203 INFO:tasks.rados.rados.0.smithi017.stdout:558: writing smithi017107884-7 from 588311 to 1203643 tid 1 2023-03-29T14:04:33.210 INFO:tasks.rados.rados.0.smithi017.stdout:558: writing smithi017107884-7 from 1842290 to 2405049 tid 2 2023-03-29T14:04:33.210 INFO:tasks.rados.rados.0.smithi017.stdout:559: write oid 24 current snap is 70 2023-03-29T14:04:33.210 INFO:tasks.rados.rados.0.smithi017.stdout:559: seq_num 274 ranges {605118=406934,1617125=706417} 2023-03-29T14:04:33.215 INFO:tasks.rados.rados.0.smithi017.stdout:559: writing smithi017107884-24 from 605118 to 1012052 tid 1 2023-03-29T14:04:33.223 INFO:tasks.rados.rados.0.smithi017.stdout:559: writing smithi017107884-24 from 1617125 to 2323542 tid 2 2023-03-29T14:04:33.223 INFO:tasks.rados.rados.0.smithi017.stdout:560: read oid 20 snap 69 2023-03-29T14:04:33.223 INFO:tasks.rados.rados.0.smithi017.stdout:560: expect (ObjNum 153 snap 21 seq_num 153) 2023-03-29T14:04:33.223 INFO:tasks.rados.rados.0.smithi017.stdout:561: snap_remove snap 50 2023-03-29T14:04:33.471 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:33.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:33.637+0000 7fa7be5ff700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:33.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:33.641+0000 7fa7bd39d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:34.200 INFO:tasks.rados.rados.0.smithi017.stdout:561: done (4 left) 2023-03-29T14:04:34.200 INFO:tasks.rados.rados.0.smithi017.stdout:562: snap_create 2023-03-29T14:04:34.201 INFO:tasks.rados.rados.0.smithi017.stdout:557: expect (ObjNum 252 snap 70 seq_num 252) 2023-03-29T14:04:34.306 INFO:tasks.rados.rados.0.smithi017.stdout:559: finishing write tid 1 to smithi017107884-24 2023-03-29T14:04:34.306 INFO:tasks.rados.rados.0.smithi017.stdout:558: finishing write tid 1 to smithi017107884-7 2023-03-29T14:04:34.306 INFO:tasks.rados.rados.0.smithi017.stdout:560: expect (ObjNum 153 snap 21 seq_num 153) 2023-03-29T14:04:34.478 INFO:tasks.rados.rados.0.smithi017.stdout:558: finishing write tid 2 to smithi017107884-7 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:559: finishing write tid 2 to smithi017107884-24 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:558: finishing write tid 3 to smithi017107884-7 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:558: finishing write tid 4 to smithi017107884-7 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:558: oid 7 updating version 0 to 46 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:558: oid 7 version 46 is already newer than 46 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:558: oid 7 version 46 is already newer than 46 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 46 (ObjNum 273 snap 70 seq_num 273) dirty exists 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:558: left oid 7 (ObjNum 273 snap 70 seq_num 273) 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:559: finishing write tid 3 to smithi017107884-24 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:559: finishing write tid 4 to smithi017107884-24 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:559: oid 24 updating version 0 to 49 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:559: oid 24 version 49 is already newer than 49 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:559: oid 24 version 49 is already newer than 49 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 49 (ObjNum 274 snap 70 seq_num 274) dirty exists 2023-03-29T14:04:34.479 INFO:tasks.rados.rados.0.smithi017.stdout:559: left oid 24 (ObjNum 274 snap 70 seq_num 274) 2023-03-29T14:04:35.201 INFO:tasks.rados.rados.0.smithi017.stdout:557: done (4 left) 2023-03-29T14:04:35.202 INFO:tasks.rados.rados.0.smithi017.stdout:558: done (3 left) 2023-03-29T14:04:35.202 INFO:tasks.rados.rados.0.smithi017.stdout:559: done (2 left) 2023-03-29T14:04:35.202 INFO:tasks.rados.rados.0.smithi017.stdout:560: done (1 left) 2023-03-29T14:04:35.202 INFO:tasks.rados.rados.0.smithi017.stdout:562: done (0 left) 2023-03-29T14:04:35.202 INFO:tasks.rados.rados.0.smithi017.stdout:563: write oid 6 current snap is 71 2023-03-29T14:04:35.202 INFO:tasks.rados.rados.0.smithi017.stdout:563: seq_num 275 ranges {621925=482183,1559254=566428} 2023-03-29T14:04:35.207 INFO:tasks.rados.rados.0.smithi017.stdout:563: writing smithi017107884-6 from 621925 to 1104108 tid 1 2023-03-29T14:04:35.213 INFO:tasks.rados.rados.0.smithi017.stdout:563: writing smithi017107884-6 from 1559254 to 2125682 tid 2 2023-03-29T14:04:35.213 INFO:tasks.rados.rados.0.smithi017.stdout:564: snap_create 2023-03-29T14:04:35.329 INFO:tasks.rados.rados.0.smithi017.stdout:563: finishing write tid 1 to smithi017107884-6 2023-03-29T14:04:35.330 INFO:tasks.rados.rados.0.smithi017.stdout:563: finishing write tid 2 to smithi017107884-6 2023-03-29T14:04:35.331 INFO:tasks.rados.rados.0.smithi017.stdout:563: finishing write tid 3 to smithi017107884-6 2023-03-29T14:04:35.331 INFO:tasks.rados.rados.0.smithi017.stdout:563: finishing write tid 4 to smithi017107884-6 2023-03-29T14:04:35.332 INFO:tasks.rados.rados.0.smithi017.stdout:563: oid 6 updating version 0 to 60 2023-03-29T14:04:35.332 INFO:tasks.rados.rados.0.smithi017.stdout:563: oid 6 version 60 is already newer than 60 2023-03-29T14:04:35.332 INFO:tasks.rados.rados.0.smithi017.stdout:563: oid 6 version 60 is already newer than 60 2023-03-29T14:04:35.332 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 60 (ObjNum 275 snap 71 seq_num 275) dirty exists 2023-03-29T14:04:35.332 INFO:tasks.rados.rados.0.smithi017.stdout:563: left oid 6 (ObjNum 275 snap 71 seq_num 275) 2023-03-29T14:04:35.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:35.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:35.855+0000 7f438b1b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:35.860 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:35.860+0000 7f438b1b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:36.202 INFO:tasks.rados.rados.0.smithi017.stdout:563: done (1 left) 2023-03-29T14:04:36.202 INFO:tasks.rados.rados.0.smithi017.stdout:564: done (0 left) 2023-03-29T14:04:36.202 INFO:tasks.rados.rados.0.smithi017.stdout:565: write oid 22 current snap is 72 2023-03-29T14:04:36.202 INFO:tasks.rados.rados.0.smithi017.stdout:565: seq_num 276 ranges {638732=557432,1617736=710086} 2023-03-29T14:04:36.208 INFO:tasks.rados.rados.0.smithi017.stdout:565: writing smithi017107884-22 from 638732 to 1196164 tid 1 2023-03-29T14:04:36.215 INFO:tasks.rados.rados.0.smithi017.stdout:565: writing smithi017107884-22 from 1617736 to 2327822 tid 2 2023-03-29T14:04:36.215 INFO:tasks.rados.rados.0.smithi017.stdout:566: snap_remove snap 69 2023-03-29T14:04:36.389 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:36.389 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:36.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:36.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:36.519+0000 7f668920a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:36.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:36.526+0000 7f668920a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:36.733 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:36.735 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:36.747 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-03-29T14:04:36.200858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":208949860,"num_objects":366,"num_object_clones":262,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":28,"num_read":1531,"num_read_kb":793969,"num_write":2100,"num_write_kb":531942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":992,"ondisk_log_size":992,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":720432,"kb_used_data":673796,"kb_used_omap":0,"kb_used_meta":45056,"kb_avail":376766928,"statfs":{"total":386547056640,"available":385809334272,"internally_reserved":0,"allocated":689967104,"data_stored":829292764,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":46137344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.905722"},"pg_stats":[{"pgid":"3.f","version":"142'29","reported_seq":177,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.327468+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:30.327468+0000","last_peered":"2023-03-29T14:04:30.327468+0000","last_clean":"2023-03-29T14:04:30.327468+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:30.327468+0000","last_undegraded":"2023-03-29T14:04:30.327468+0000","last_fullsized":"2023-03-29T14:04:30.327468+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4470728,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":75,"num_read_kb":50194,"num_write":60,"num_write_kb":15306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":143,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:29.295284+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:29.295284+0000","last_peered":"2023-03-29T14:04:29.295284+0000","last_clean":"2023-03-29T14:04:29.295284+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:29.295284+0000","last_undegraded":"2023-03-29T14:04:29.295284+0000","last_fullsized":"2023-03-29T14:04:29.295284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"129'87","reported_seq":299,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.643875+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:30.643875+0000","last_peered":"2023-03-29T14:04:30.643875+0000","last_clean":"2023-03-29T14:04:30.643875+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:30.643875+0000","last_undegraded":"2023-03-29T14:04:30.643875+0000","last_fullsized":"2023-03-29T14:04:30.643875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269110,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":100,"num_read_kb":56304,"num_write":174,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"145'83","reported_seq":293,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.756089+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:30.756089+0000","last_peered":"2023-03-29T14:04:30.756089+0000","last_clean":"2023-03-29T14:04:30.756089+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:30.756089+0000","last_undegraded":"2023-03-29T14:04:30.756089+0000","last_fullsized":"2023-03-29T14:04:30.756089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185306,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":114,"num_read_kb":55359,"num_write":168,"num_write_kb":46564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"142'71","reported_seq":277,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:29.339791+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:29.339791+0000","last_peered":"2023-03-29T14:04:29.339791+0000","last_clean":"2023-03-29T14:04:29.339791+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:29.339791+0000","last_undegraded":"2023-03-29T14:04:29.339791+0000","last_fullsized":"2023-03-29T14:04:29.339791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9311144,"num_objects":32,"num_object_clones":24,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":116,"num_read_kb":64813,"num_write":142,"num_write_kb":42860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":109,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.641442+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:30.641442+0000","last_peered":"2023-03-29T14:04:30.641442+0000","last_clean":"2023-03-29T14:04:30.641442+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:30.641442+0000","last_undegraded":"2023-03-29T14:04:30.641442+0000","last_fullsized":"2023-03-29T14:04:30.641442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"139'53","reported_seq":234,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.648248+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:30.648248+0000","last_peered":"2023-03-29T14:04:30.648248+0000","last_clean":"2023-03-29T14:04:30.648248+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:30.648248+0000","last_undegraded":"2023-03-29T14:04:30.648248+0000","last_fullsized":"2023-03-29T14:04:30.648248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697738,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":86,"num_read_kb":47167,"num_write":106,"num_write_kb":29126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"145'43","reported_seq":208,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.753414+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:30.753414+0000","last_peered":"2023-03-29T14:04:30.753414+0000","last_clean":"2023-03-29T14:04:30.753414+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:30.753414+0000","last_undegraded":"2023-03-29T14:04:30.753414+0000","last_fullsized":"2023-03-29T14:04:30.753414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11092686,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":34278,"num_write":92,"num_write_kb":26114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":114,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:29.303411+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:29.303411+0000","last_peered":"2023-03-29T14:04:29.303411+0000","last_clean":"2023-03-29T14:04:29.303411+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:29.303411+0000","last_undegraded":"2023-03-29T14:04:29.303411+0000","last_fullsized":"2023-03-29T14:04:29.303411+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'33","reported_seq":160,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:28.310248+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:28.310248+0000","last_peered":"2023-03-29T14:04:28.310248+0000","last_clean":"2023-03-29T14:04:28.310248+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:28.310248+0000","last_undegraded":"2023-03-29T14:04:28.310248+0000","last_fullsized":"2023-03-29T14:04:28.310248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":42,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":112,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.647599+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:30.647599+0000","last_peered":"2023-03-29T14:04:30.647599+0000","last_clean":"2023-03-29T14:04:30.647599+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:30.647599+0000","last_undegraded":"2023-03-29T14:04:30.647599+0000","last_fullsized":"2023-03-29T14:04:30.647599+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"120'45","reported_seq":214,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.014284+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:30.014284+0000","last_peered":"2023-03-29T14:04:30.014284+0000","last_clean":"2023-03-29T14:04:30.014284+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:30.014284+0000","last_undegraded":"2023-03-29T14:04:30.014284+0000","last_fullsized":"2023-03-29T14:04:30.014284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462440,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":31044,"num_write":94,"num_write_kb":26534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:28.305762+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:28.305762+0000","last_peered":"2023-03-29T14:04:28.305762+0000","last_clean":"2023-03-29T14:04:28.305762+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:28.305762+0000","last_undegraded":"2023-03-29T14:04:28.305762+0000","last_fullsized":"2023-03-29T14:04:28.305762+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":257,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:29.310495+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:29.310495+0000","last_peered":"2023-03-29T14:04:29.310495+0000","last_clean":"2023-03-29T14:04:29.310495+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:29.310495+0000","last_undegraded":"2023-03-29T14:04:29.310495+0000","last_fullsized":"2023-03-29T14:04:29.310495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":112,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.639203+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:30.639203+0000","last_peered":"2023-03-29T14:04:30.639203+0000","last_clean":"2023-03-29T14:04:30.639203+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:30.639203+0000","last_undegraded":"2023-03-29T14:04:30.639203+0000","last_fullsized":"2023-03-29T14:04:30.639203+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":339,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:29.285823+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:29.285823+0000","last_peered":"2023-03-29T14:04:29.285823+0000","last_clean":"2023-03-29T14:04:29.285823+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:29.285823+0000","last_undegraded":"2023-03-29T14:04:29.285823+0000","last_fullsized":"2023-03-29T14:04:29.285823+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"144'56","reported_seq":262,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.646953+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:30.646953+0000","last_peered":"2023-03-29T14:04:30.646953+0000","last_clean":"2023-03-29T14:04:30.646953+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:30.646953+0000","last_undegraded":"2023-03-29T14:04:30.646953+0000","last_fullsized":"2023-03-29T14:04:30.646953+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15428826,"num_objects":22,"num_object_clones":14,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":133,"num_read_kb":84069,"num_write":116,"num_write_kb":33354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":112,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.635810+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:30.635810+0000","last_peered":"2023-03-29T14:04:30.635810+0000","last_clean":"2023-03-29T14:04:30.635810+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:30.635810+0000","last_undegraded":"2023-03-29T14:04:30.635810+0000","last_fullsized":"2023-03-29T14:04:30.635810+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"145'126","reported_seq":407,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.620005+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:30.620005+0000","last_peered":"2023-03-29T14:04:30.620005+0000","last_clean":"2023-03-29T14:04:30.620005+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:30.620005+0000","last_undegraded":"2023-03-29T14:04:30.620005+0000","last_fullsized":"2023-03-29T14:04:30.620005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026118,"num_objects":42,"num_object_clones":28,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":183,"num_read_kb":106523,"num_write":258,"num_write_kb":72600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":114,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:29.306923+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:29.306923+0000","last_peered":"2023-03-29T14:04:29.306923+0000","last_clean":"2023-03-29T14:04:29.306923+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:29.306923+0000","last_undegraded":"2023-03-29T14:04:29.306923+0000","last_fullsized":"2023-03-29T14:04:29.306923+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"139'41","reported_seq":213,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.546178+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:30.546178+0000","last_peered":"2023-03-29T14:04:30.546178+0000","last_clean":"2023-03-29T14:04:30.546178+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:30.546178+0000","last_undegraded":"2023-03-29T14:04:30.546178+0000","last_fullsized":"2023-03-29T14:04:30.546178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437114,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":70,"num_read_kb":44595,"num_write":90,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"145'92","reported_seq":315,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.575533+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:30.575533+0000","last_peered":"2023-03-29T14:04:30.575533+0000","last_clean":"2023-03-29T14:04:30.575533+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:30.575533+0000","last_undegraded":"2023-03-29T14:04:30.575533+0000","last_fullsized":"2023-03-29T14:04:30.575533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21445930,"num_objects":36,"num_object_clones":26,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":64144,"num_write":186,"num_write_kb":56676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:29.313721+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:29.313721+0000","last_peered":"2023-03-29T14:04:29.313721+0000","last_clean":"2023-03-29T14:04:29.313721+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:29.313721+0000","last_undegraded":"2023-03-29T14:04:29.313721+0000","last_fullsized":"2023-03-29T14:04:29.313721+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":112,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.640869+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:30.640869+0000","last_peered":"2023-03-29T14:04:30.640869+0000","last_clean":"2023-03-29T14:04:30.640869+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:30.640869+0000","last_undegraded":"2023-03-29T14:04:30.640869+0000","last_fullsized":"2023-03-29T14:04:30.640869+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"144'61","reported_seq":263,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-03-29T14:04:30.153176+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:30.153176+0000","last_peered":"2023-03-29T14:04:30.153176+0000","last_clean":"2023-03-29T14:04:30.153176+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:30.153176+0000","last_undegraded":"2023-03-29T14:04:30.153176+0000","last_fullsized":"2023-03-29T14:04:30.153176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18588542,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":112,"num_read_kb":67582,"num_write":124,"num_write_kb":35700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":207769124,"num_objects":362,"num_object_clones":262,"num_object_copies":1086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":28,"num_read":1389,"num_read_kb":793849,"num_write":1826,"num_write_kb":528154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":895,"ondisk_log_size":895,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":189460,"kb_used_data":176776,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94182380,"statfs":{"total":96636764160,"available":96442757120,"internally_reserved":0,"allocated":181018624,"data_stored":217703100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":127764,"kb_used_data":120268,"kb_used_omap":0,"kb_used_meta":7488,"kb_avail":94244076,"statfs":{"total":96636764160,"available":96505933824,"internally_reserved":0,"allocated":123154432,"data_stored":143495061,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7667712},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705808,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":208124,"kb_used_data":194736,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94163716,"statfs":{"total":96636764160,"available":96423645184,"internally_reserved":0,"allocated":199409664,"data_stored":241943966,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705808,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":195084,"kb_used_data":182016,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94176756,"statfs":{"total":96636764160,"available":96436998144,"internally_reserved":0,"allocated":186384384,"data_stored":226150637,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:37.204 INFO:tasks.rados.rados.0.smithi017.stdout:566: done (1 left) 2023-03-29T14:04:37.204 INFO:tasks.rados.rados.0.smithi017.stdout:567: snap_create 2023-03-29T14:04:37.204 INFO:tasks.rados.rados.0.smithi017.stdout:565: finishing write tid 1 to smithi017107884-22 2023-03-29T14:04:37.204 INFO:tasks.rados.rados.0.smithi017.stdout:565: finishing write tid 2 to smithi017107884-22 2023-03-29T14:04:37.204 INFO:tasks.rados.rados.0.smithi017.stdout:565: finishing write tid 3 to smithi017107884-22 2023-03-29T14:04:37.204 INFO:tasks.rados.rados.0.smithi017.stdout:565: finishing write tid 4 to smithi017107884-22 2023-03-29T14:04:37.204 INFO:tasks.rados.rados.0.smithi017.stdout:565: oid 22 updating version 0 to 35 2023-03-29T14:04:37.205 INFO:tasks.rados.rados.0.smithi017.stdout:565: oid 22 version 35 is already newer than 35 2023-03-29T14:04:37.205 INFO:tasks.rados.rados.0.smithi017.stdout:565: oid 22 version 35 is already newer than 35 2023-03-29T14:04:37.205 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 35 (ObjNum 276 snap 72 seq_num 276) dirty exists 2023-03-29T14:04:37.205 INFO:tasks.rados.rados.0.smithi017.stdout:565: left oid 22 (ObjNum 276 snap 72 seq_num 276) 2023-03-29T14:04:37.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:38.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:38.072+0000 7f456fe44700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:38.077 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:38.077+0000 7f456fe44700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:38.204 INFO:tasks.rados.rados.0.smithi017.stdout:565: done (1 left) 2023-03-29T14:04:38.204 INFO:tasks.rados.rados.0.smithi017.stdout:567: done (0 left) 2023-03-29T14:04:38.204 INFO:tasks.rados.rados.0.smithi017.stdout:568: snap_create 2023-03-29T14:04:39.206 INFO:tasks.rados.rados.0.smithi017.stdout:568: done (0 left) 2023-03-29T14:04:39.206 INFO:tasks.rados.rados.0.smithi017.stdout:569: snap_remove snap 51 2023-03-29T14:04:40.150 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:40.207 INFO:tasks.rados.rados.0.smithi017.stdout:569: done (0 left) 2023-03-29T14:04:40.207 INFO:tasks.rados.rados.0.smithi017.stdout:570: write oid 46 current snap is 74 2023-03-29T14:04:40.207 INFO:tasks.rados.rados.0.smithi017.stdout:570: seq_num 277 ranges {655539=632681,2076218=570097} 2023-03-29T14:04:40.216 INFO:tasks.rados.rados.0.smithi017.stdout:570: writing smithi017107884-46 from 655539 to 1288220 tid 1 2023-03-29T14:04:40.221 INFO:tasks.rados.rados.0.smithi017.stdout:570: writing smithi017107884-46 from 2076218 to 2646315 tid 2 2023-03-29T14:04:40.221 INFO:tasks.rados.rados.0.smithi017.stdout:571: snap_remove snap 57 2023-03-29T14:04:40.333 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:40.332+0000 7fdd184f4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:40.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:40.336+0000 7fdd184f4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:41.210 INFO:tasks.rados.rados.0.smithi017.stdout:571: done (1 left) 2023-03-29T14:04:41.210 INFO:tasks.rados.rados.0.smithi017.stdout:572: delete oid 37 current snap is 74 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: finishing write tid 1 to smithi017107884-46 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: finishing write tid 2 to smithi017107884-46 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: finishing write tid 3 to smithi017107884-46 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: finishing write tid 4 to smithi017107884-46 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: oid 46 updating version 0 to 75 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: oid 46 version 75 is already newer than 75 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: oid 46 version 75 is already newer than 75 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 75 (ObjNum 277 snap 74 seq_num 277) dirty exists 2023-03-29T14:04:41.211 INFO:tasks.rados.rados.0.smithi017.stdout:570: left oid 46 (ObjNum 277 snap 74 seq_num 277) 2023-03-29T14:04:41.265 INFO:tasks.rados.rados.0.smithi017.stdout:570: done (1 left) 2023-03-29T14:04:41.265 INFO:tasks.rados.rados.0.smithi017.stdout:572: done (0 left) 2023-03-29T14:04:41.265 INFO:tasks.rados.rados.0.smithi017.stdout:573: delete oid 48 current snap is 74 2023-03-29T14:04:41.282 INFO:tasks.rados.rados.0.smithi017.stdout:573: done (0 left) 2023-03-29T14:04:41.283 INFO:tasks.rados.rados.0.smithi017.stdout:574: read oid 9 snap -1 2023-03-29T14:04:41.283 INFO:tasks.rados.rados.0.smithi017.stdout:574: expect (ObjNum 256 snap 70 seq_num 256) 2023-03-29T14:04:41.283 INFO:tasks.rados.rados.0.smithi017.stdout:575: snap_create 2023-03-29T14:04:41.298 INFO:tasks.rados.rados.0.smithi017.stdout:574: expect (ObjNum 256 snap 70 seq_num 256) 2023-03-29T14:04:41.756 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:41.756 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:41.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:41.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:41.885+0000 7f4cd8e49700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:41.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:41.892+0000 7f4cd8e49700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:42.097 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:42.101 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:42.111 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2023-03-29T14:04:41.208979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203807116,"num_objects":378,"num_object_clones":274,"num_object_copies":1134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":28,"num_read":1572,"num_read_kb":819431,"num_write":2156,"num_write_kb":548340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1024,"ondisk_log_size":1024,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":746608,"kb_used_data":700880,"kb_used_omap":0,"kb_used_meta":45696,"kb_avail":376740752,"statfs":{"total":386547056640,"available":385782530048,"internally_reserved":0,"allocated":717701120,"data_stored":865240887,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":46792704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":806736,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":10006,"num_write":8,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.007039"},"pg_stats":[{"pgid":"3.f","version":"151'35","reported_seq":195,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-03-29T14:04:36.276635+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:36.276635+0000","last_peered":"2023-03-29T14:04:36.276635+0000","last_clean":"2023-03-29T14:04:36.276635+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:36.276635+0000","last_undegraded":"2023-03-29T14:04:36.276635+0000","last_fullsized":"2023-03-29T14:04:36.276635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":90,"num_read_kb":60200,"num_write":68,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":147,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.429201+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:34.429201+0000","last_peered":"2023-03-29T14:04:34.429201+0000","last_clean":"2023-03-29T14:04:34.429201+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:34.429201+0000","last_undegraded":"2023-03-29T14:04:34.429201+0000","last_fullsized":"2023-03-29T14:04:34.429201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"147'90","reported_seq":310,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.449751+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:34.449751+0000","last_peered":"2023-03-29T14:04:34.449751+0000","last_clean":"2023-03-29T14:04:34.449751+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:34.449751+0000","last_undegraded":"2023-03-29T14:04:34.449751+0000","last_fullsized":"2023-03-29T14:04:34.449751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378504,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":56305,"num_write":182,"num_write_kb":54224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"145'83","reported_seq":297,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.441122+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:34.441122+0000","last_peered":"2023-03-29T14:04:34.441122+0000","last_clean":"2023-03-29T14:04:34.441122+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:34.441122+0000","last_undegraded":"2023-03-29T14:04:34.441122+0000","last_fullsized":"2023-03-29T14:04:34.441122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185306,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":114,"num_read_kb":55359,"num_write":168,"num_write_kb":46564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"142'71","reported_seq":284,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.444316+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:34.444316+0000","last_peered":"2023-03-29T14:04:34.444316+0000","last_clean":"2023-03-29T14:04:34.444316+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:34.444316+0000","last_undegraded":"2023-03-29T14:04:34.444316+0000","last_fullsized":"2023-03-29T14:04:34.444316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9311144,"num_objects":32,"num_object_clones":24,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":123,"num_read_kb":71222,"num_write":142,"num_write_kb":42860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":113,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.430045+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:34.430045+0000","last_peered":"2023-03-29T14:04:34.430045+0000","last_clean":"2023-03-29T14:04:34.430045+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:34.430045+0000","last_undegraded":"2023-03-29T14:04:34.430045+0000","last_fullsized":"2023-03-29T14:04:34.430045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"139'53","reported_seq":238,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.446908+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:34.446908+0000","last_peered":"2023-03-29T14:04:34.446908+0000","last_clean":"2023-03-29T14:04:34.446908+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:34.446908+0000","last_undegraded":"2023-03-29T14:04:34.446908+0000","last_fullsized":"2023-03-29T14:04:34.446908+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697738,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":86,"num_read_kb":47167,"num_write":106,"num_write_kb":29126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"148'46","reported_seq":219,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.445668+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:34.445668+0000","last_peered":"2023-03-29T14:04:34.445668+0000","last_clean":"2023-03-29T14:04:34.445668+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:34.445668+0000","last_undegraded":"2023-03-29T14:04:34.445668+0000","last_fullsized":"2023-03-29T14:04:34.445668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11294370,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":34279,"num_write":100,"num_write_kb":28416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.432269+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:34.432269+0000","last_peered":"2023-03-29T14:04:34.432269+0000","last_clean":"2023-03-29T14:04:34.432269+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:34.432269+0000","last_undegraded":"2023-03-29T14:04:34.432269+0000","last_fullsized":"2023-03-29T14:04:34.432269+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'33","reported_seq":165,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-03-29T14:04:35.245185+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:35.245185+0000","last_peered":"2023-03-29T14:04:35.245185+0000","last_clean":"2023-03-29T14:04:35.245185+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:35.245185+0000","last_undegraded":"2023-03-29T14:04:35.245185+0000","last_fullsized":"2023-03-29T14:04:35.245185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":42,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.440514+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:34.440514+0000","last_peered":"2023-03-29T14:04:34.440514+0000","last_clean":"2023-03-29T14:04:34.440514+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:34.440514+0000","last_undegraded":"2023-03-29T14:04:34.440514+0000","last_fullsized":"2023-03-29T14:04:34.440514+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"148'49","reported_seq":229,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.222342+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:34.222342+0000","last_peered":"2023-03-29T14:04:34.222342+0000","last_clean":"2023-03-29T14:04:34.222342+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:34.222342+0000","last_undegraded":"2023-03-29T14:04:34.222342+0000","last_fullsized":"2023-03-29T14:04:34.222342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12840614,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":40086,"num_write":102,"num_write_kb":28710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-03-29T14:04:35.263817+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:35.263817+0000","last_peered":"2023-03-29T14:04:35.263817+0000","last_clean":"2023-03-29T14:04:35.263817+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:35.263817+0000","last_undegraded":"2023-03-29T14:04:35.263817+0000","last_fullsized":"2023-03-29T14:04:35.263817+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":261,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.440036+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:34.440036+0000","last_peered":"2023-03-29T14:04:34.440036+0000","last_clean":"2023-03-29T14:04:34.440036+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:34.440036+0000","last_undegraded":"2023-03-29T14:04:34.440036+0000","last_fullsized":"2023-03-29T14:04:34.440036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":116,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.441792+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:34.441792+0000","last_peered":"2023-03-29T14:04:34.441792+0000","last_clean":"2023-03-29T14:04:34.441792+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:34.441792+0000","last_undegraded":"2023-03-29T14:04:34.441792+0000","last_fullsized":"2023-03-29T14:04:34.441792+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":343,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.431679+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:34.431679+0000","last_peered":"2023-03-29T14:04:34.431679+0000","last_clean":"2023-03-29T14:04:34.431679+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:34.431679+0000","last_undegraded":"2023-03-29T14:04:34.431679+0000","last_fullsized":"2023-03-29T14:04:34.431679+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"150'60","reported_seq":273,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:35.283552+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:35.283552+0000","last_peered":"2023-03-29T14:04:35.283552+0000","last_clean":"2023-03-29T14:04:35.283552+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:35.283552+0000","last_undegraded":"2023-03-29T14:04:35.283552+0000","last_fullsized":"2023-03-29T14:04:35.283552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15966650,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":134,"num_read_kb":84070,"num_write":124,"num_write_kb":35404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":116,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.442327+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:34.442327+0000","last_peered":"2023-03-29T14:04:34.442327+0000","last_clean":"2023-03-29T14:04:34.442327+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:34.442327+0000","last_undegraded":"2023-03-29T14:04:34.442327+0000","last_fullsized":"2023-03-29T14:04:34.442327+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"146'134","reported_seq":426,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.214854+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:34.214854+0000","last_peered":"2023-03-29T14:04:34.214854+0000","last_clean":"2023-03-29T14:04:34.214854+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:34.214854+0000","last_undegraded":"2023-03-29T14:04:34.214854+0000","last_fullsized":"2023-03-29T14:04:34.214854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37614396,"num_objects":46,"num_object_clones":32,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":106525,"num_write":274,"num_write_kb":77992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.441246+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:34.441246+0000","last_peered":"2023-03-29T14:04:34.441246+0000","last_clean":"2023-03-29T14:04:34.441246+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:34.441246+0000","last_undegraded":"2023-03-29T14:04:34.441246+0000","last_fullsized":"2023-03-29T14:04:34.441246+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"145'43","reported_seq":217,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.448156+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:34.448156+0000","last_peered":"2023-03-29T14:04:34.448156+0000","last_clean":"2023-03-29T14:04:34.448156+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:34.448156+0000","last_undegraded":"2023-03-29T14:04:34.448156+0000","last_fullsized":"2023-03-29T14:04:34.448156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437114,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":70,"num_read_kb":44595,"num_write":90,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'94","reported_seq":321,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.443716+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:34.443716+0000","last_peered":"2023-03-29T14:04:34.443716+0000","last_clean":"2023-03-29T14:04:34.443716+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:34.443716+0000","last_undegraded":"2023-03-29T14:04:34.443716+0000","last_fullsized":"2023-03-29T14:04:34.443716+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13681096,"num_objects":38,"num_object_clones":28,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":122,"num_read_kb":64144,"num_write":186,"num_write_kb":56676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.449225+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:34.449225+0000","last_peered":"2023-03-29T14:04:34.449225+0000","last_clean":"2023-03-29T14:04:34.449225+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:34.449225+0000","last_undegraded":"2023-03-29T14:04:34.449225+0000","last_fullsized":"2023-03-29T14:04:34.449225+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":116,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.439903+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:34.439903+0000","last_peered":"2023-03-29T14:04:34.439903+0000","last_clean":"2023-03-29T14:04:34.439903+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:34.439903+0000","last_undegraded":"2023-03-29T14:04:34.439903+0000","last_fullsized":"2023-03-29T14:04:34.439903+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"144'61","reported_seq":268,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-03-29T14:04:34.216009+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:34.216009+0000","last_peered":"2023-03-29T14:04:34.216009+0000","last_clean":"2023-03-29T14:04:34.216009+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:34.216009+0000","last_undegraded":"2023-03-29T14:04:34.216009+0000","last_fullsized":"2023-03-29T14:04:34.216009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18588542,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":112,"num_read_kb":67582,"num_write":124,"num_write_kb":35700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":202626380,"num_objects":374,"num_object_clones":274,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":28,"num_read":1430,"num_read_kb":819311,"num_write":1882,"num_write_kb":544552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":927,"ondisk_log_size":927,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":194776,"kb_used_data":182480,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94177064,"statfs":{"total":96636764160,"available":96437313536,"internally_reserved":0,"allocated":186859520,"data_stored":225382495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131300,"kb_used_data":123676,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94240540,"statfs":{"total":96636764160,"available":96502312960,"internally_reserved":0,"allocated":126644224,"data_stored":148658179,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":218740,"kb_used_data":205676,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94153100,"statfs":{"total":96636764160,"available":96412774400,"internally_reserved":0,"allocated":210612224,"data_stored":255581258,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":201792,"kb_used_data":189048,"kb_used_omap":0,"kb_used_meta":12736,"kb_avail":94170048,"statfs":{"total":96636764160,"available":96430129152,"internally_reserved":0,"allocated":193585152,"data_stored":235618955,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13041664},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:42.218 INFO:tasks.rados.rados.0.smithi017.stdout:574: done (1 left) 2023-03-29T14:04:42.218 INFO:tasks.rados.rados.0.smithi017.stdout:575: done (0 left) 2023-03-29T14:04:42.218 INFO:tasks.rados.rados.0.smithi017.stdout:576: write oid 45 current snap is 75 2023-03-29T14:04:42.218 INFO:tasks.rados.rados.0.smithi017.stdout:576: seq_num 280 ranges {705960=458428,1735311=717424} 2023-03-29T14:04:42.220 INFO:tasks.rados.rados.0.smithi017.stdout:576: writing smithi017107884-45 from 705960 to 1164388 tid 1 2023-03-29T14:04:42.227 INFO:tasks.rados.rados.0.smithi017.stdout:576: writing smithi017107884-45 from 1735311 to 2452735 tid 2 2023-03-29T14:04:42.227 INFO:tasks.rados.rados.0.smithi017.stdout:577: read oid 2 snap 67 2023-03-29T14:04:42.228 INFO:tasks.rados.rados.0.smithi017.stdout:577: expect deleted 2023-03-29T14:04:42.228 INFO:tasks.rados.rados.0.smithi017.stdout:578: snap_remove snap 68 2023-03-29T14:04:42.430 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:42.560 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:42.559+0000 7fa5aa58e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:42.564 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:42.564+0000 7fa5aa58e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:43.198 INFO:tasks.rados.rados.0.smithi017.stdout:578: done (2 left) 2023-03-29T14:04:43.199 INFO:tasks.rados.rados.0.smithi017.stdout:579: snap_remove snap 74 2023-03-29T14:04:44.204 INFO:tasks.rados.rados.0.smithi017.stdout:579: done (2 left) 2023-03-29T14:04:44.204 INFO:tasks.rados.rados.0.smithi017.stdout:580: write oid 19 current snap is 75 2023-03-29T14:04:44.204 INFO:tasks.rados.rados.0.smithi017.stdout:580: seq_num 281 ranges {722767=533677,1793793=461082} 2023-03-29T14:04:44.211 INFO:tasks.rados.rados.0.smithi017.stdout:580: writing smithi017107884-19 from 722767 to 1256444 tid 1 2023-03-29T14:04:44.217 INFO:tasks.rados.rados.0.smithi017.stdout:580: writing smithi017107884-19 from 1793793 to 2254875 tid 2 2023-03-29T14:04:44.217 INFO:tasks.rados.rados.0.smithi017.stdout:581: write oid 16 current snap is 75 2023-03-29T14:04:44.217 INFO:tasks.rados.rados.0.smithi017.stdout:581: seq_num 282 ranges {739574=725279,1968628=721093} 2023-03-29T14:04:44.225 INFO:tasks.rados.rados.0.smithi017.stdout:581: writing smithi017107884-16 from 739574 to 1464853 tid 1 2023-03-29T14:04:44.235 INFO:tasks.rados.rados.0.smithi017.stdout:581: writing smithi017107884-16 from 1968628 to 2689721 tid 2 2023-03-29T14:04:44.235 INFO:tasks.rados.rados.0.smithi017.stdout:582: read oid 18 snap 53 2023-03-29T14:04:44.235 INFO:tasks.rados.rados.0.smithi017.stdout:582: expect (ObjNum 17 snap 0 seq_num 17) 2023-03-29T14:04:44.235 INFO:tasks.rados.rados.0.smithi017.stdout:583: write oid 8 current snap is 75 2023-03-29T14:04:44.235 INFO:tasks.rados.rados.0.smithi017.stdout:583: seq_num 283 ranges {756381=400528,1910757=464751} 2023-03-29T14:04:44.239 INFO:tasks.rados.rados.0.smithi017.stdout:583: writing smithi017107884-8 from 756381 to 1156909 tid 1 2023-03-29T14:04:44.246 INFO:tasks.rados.rados.0.smithi017.stdout:583: writing smithi017107884-8 from 1910757 to 2375508 tid 2 2023-03-29T14:04:44.246 INFO:tasks.rados.rados.0.smithi017.stdout:584: write (excl) oid 20 current snap is 75 2023-03-29T14:04:44.246 INFO:tasks.rados.rados.0.smithi017.stdout:584: seq_num 284 ranges {773188=475777,1969239=724762} 2023-03-29T14:04:44.253 INFO:tasks.rados.rados.0.smithi017.stdout:584: writing smithi017107884-20 from 773188 to 1248965 tid 1 2023-03-29T14:04:44.263 INFO:tasks.rados.rados.0.smithi017.stdout:584: writing smithi017107884-20 from 1969239 to 2694001 tid 2 2023-03-29T14:04:44.263 INFO:tasks.rados.rados.0.smithi017.stdout:585: snap_remove snap 42 2023-03-29T14:04:44.633 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:44.764 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:44.762+0000 7f4a9df3c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:44.767 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:44.767+0000 7f4a9df3c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:45.216 INFO:tasks.rados.rados.0.smithi017.stdout:585: done (7 left) 2023-03-29T14:04:45.216 INFO:tasks.rados.rados.0.smithi017.stdout:586: write (excl) oid 46 current snap is 75 2023-03-29T14:04:45.216 INFO:tasks.rados.rados.0.smithi017.stdout:586: seq_num 285 ranges {789995=551026,2027721=468420} 2023-03-29T14:04:45.223 INFO:tasks.rados.rados.0.smithi017.stdout:586: writing smithi017107884-46 from 789995 to 1341021 tid 1 2023-03-29T14:04:45.229 INFO:tasks.rados.rados.0.smithi017.stdout:586: writing smithi017107884-46 from 2027721 to 2496141 tid 2 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:577: done (7 left) 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:587: snap_create 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: finishing write tid 1 to smithi017107884-45 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: finishing write tid 2 to smithi017107884-45 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: finishing write tid 3 to smithi017107884-45 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: finishing write tid 4 to smithi017107884-45 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: oid 45 updating version 0 to 78 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: oid 45 version 78 is already newer than 78 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: oid 45 version 78 is already newer than 77 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 78 (ObjNum 280 snap 75 seq_num 280) dirty exists 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:576: left oid 45 (ObjNum 280 snap 75 seq_num 280) 2023-03-29T14:04:45.230 INFO:tasks.rados.rados.0.smithi017.stdout:582: expect (ObjNum 17 snap 0 seq_num 17) 2023-03-29T14:04:45.318 INFO:tasks.rados.rados.0.smithi017.stdout:581: finishing write tid 1 to smithi017107884-16 2023-03-29T14:04:45.318 INFO:tasks.rados.rados.0.smithi017.stdout:581: finishing write tid 2 to smithi017107884-16 2023-03-29T14:04:45.318 INFO:tasks.rados.rados.0.smithi017.stdout:581: finishing write tid 3 to smithi017107884-16 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:581: finishing write tid 4 to smithi017107884-16 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:581: oid 16 updating version 0 to 38 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:581: oid 16 version 38 is already newer than 38 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:581: oid 16 version 38 is already newer than 38 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 38 (ObjNum 282 snap 75 seq_num 282) dirty exists 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:581: left oid 16 (ObjNum 282 snap 75 seq_num 282) 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: finishing write tid 1 to smithi017107884-19 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:584: finishing write tid 1 to smithi017107884-20 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: finishing write tid 2 to smithi017107884-19 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: finishing write tid 3 to smithi017107884-19 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: finishing write tid 4 to smithi017107884-19 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: oid 19 updating version 0 to 65 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: oid 19 version 65 is already newer than 65 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: oid 19 version 65 is already newer than 65 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 65 (ObjNum 281 snap 75 seq_num 281) dirty exists 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:580: left oid 19 (ObjNum 281 snap 75 seq_num 281) 2023-03-29T14:04:45.319 INFO:tasks.rados.rados.0.smithi017.stdout:584: finishing write tid 2 to smithi017107884-20 2023-03-29T14:04:45.320 INFO:tasks.rados.rados.0.smithi017.stdout:584: finishing write tid 3 to smithi017107884-20 2023-03-29T14:04:45.320 INFO:tasks.rados.rados.0.smithi017.stdout:584: finishing write tid 4 to smithi017107884-20 2023-03-29T14:04:45.320 INFO:tasks.rados.rados.0.smithi017.stdout:584: oid 20 updating version 0 to 82 2023-03-29T14:04:45.320 INFO:tasks.rados.rados.0.smithi017.stdout:584: oid 20 version 82 is already newer than 82 2023-03-29T14:04:45.320 INFO:tasks.rados.rados.0.smithi017.stdout:584: oid 20 version 82 is already newer than 79 2023-03-29T14:04:45.320 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 82 (ObjNum 284 snap 75 seq_num 284) dirty exists 2023-03-29T14:04:45.320 INFO:tasks.rados.rados.0.smithi017.stdout:584: left oid 20 (ObjNum 284 snap 75 seq_num 284) 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: finishing write tid 1 to smithi017107884-8 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: finishing write tid 2 to smithi017107884-8 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: finishing write tid 3 to smithi017107884-8 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: finishing write tid 4 to smithi017107884-8 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: oid 8 updating version 0 to 68 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: oid 8 version 68 is already newer than 68 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: oid 8 version 68 is already newer than 68 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 68 (ObjNum 283 snap 75 seq_num 283) dirty exists 2023-03-29T14:04:45.321 INFO:tasks.rados.rados.0.smithi017.stdout:583: left oid 8 (ObjNum 283 snap 75 seq_num 283) 2023-03-29T14:04:45.322 INFO:tasks.rados.rados.0.smithi017.stdout:586: finishing write tid 1 to smithi017107884-46 2023-03-29T14:04:45.322 INFO:tasks.rados.rados.0.smithi017.stdout:586: finishing write tid 2 to smithi017107884-46 2023-03-29T14:04:45.358 INFO:tasks.rados.rados.0.smithi017.stdout:586: finishing write tid 3 to smithi017107884-46 2023-03-29T14:04:45.358 INFO:tasks.rados.rados.0.smithi017.stdout:586: finishing write tid 4 to smithi017107884-46 2023-03-29T14:04:45.358 INFO:tasks.rados.rados.0.smithi017.stdout:586: oid 46 updating version 0 to 83 2023-03-29T14:04:45.358 INFO:tasks.rados.rados.0.smithi017.stdout:586: oid 46 updating version 83 to 85 2023-03-29T14:04:45.358 INFO:tasks.rados.rados.0.smithi017.stdout:586: oid 46 version 85 is already newer than 85 2023-03-29T14:04:45.359 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 85 (ObjNum 285 snap 75 seq_num 285) dirty exists 2023-03-29T14:04:45.359 INFO:tasks.rados.rados.0.smithi017.stdout:586: left oid 46 (ObjNum 285 snap 75 seq_num 285) 2023-03-29T14:04:46.220 INFO:tasks.rados.rados.0.smithi017.stdout:576: done (7 left) 2023-03-29T14:04:46.220 INFO:tasks.rados.rados.0.smithi017.stdout:580: done (6 left) 2023-03-29T14:04:46.220 INFO:tasks.rados.rados.0.smithi017.stdout:581: done (5 left) 2023-03-29T14:04:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:582: done (4 left) 2023-03-29T14:04:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:583: done (3 left) 2023-03-29T14:04:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:584: done (2 left) 2023-03-29T14:04:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:586: done (1 left) 2023-03-29T14:04:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:587: done (0 left) 2023-03-29T14:04:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:588: write (excl) oid 2 current snap is 76 2023-03-29T14:04:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:588: seq_num 286 ranges {406802=626275,1686203=728431} 2023-03-29T14:04:46.226 INFO:tasks.rados.rados.0.smithi017.stdout:588: writing smithi017107884-2 from 406802 to 1033077 tid 1 2023-03-29T14:04:46.233 INFO:tasks.rados.rados.0.smithi017.stdout:588: writing smithi017107884-2 from 1686203 to 2414634 tid 2 2023-03-29T14:04:46.234 INFO:tasks.rados.rados.0.smithi017.stdout:589: read oid 35 snap -1 2023-03-29T14:04:46.234 INFO:tasks.rados.rados.0.smithi017.stdout:589: expect (ObjNum 251 snap 69 seq_num 251) 2023-03-29T14:04:46.234 INFO:tasks.rados.rados.0.smithi017.stdout:590: delete oid 8 current snap is 76 2023-03-29T14:04:46.333 INFO:tasks.rados.rados.0.smithi017.stdout:589: expect (ObjNum 251 snap 69 seq_num 251) 2023-03-29T14:04:46.422 INFO:tasks.rados.rados.0.smithi017.stdout:588: finishing write tid 1 to smithi017107884-2 2023-03-29T14:04:46.422 INFO:tasks.rados.rados.0.smithi017.stdout:588: finishing write tid 2 to smithi017107884-2 2023-03-29T14:04:46.422 INFO:tasks.rados.rados.0.smithi017.stdout:588: finishing write tid 3 to smithi017107884-2 2023-03-29T14:04:46.422 INFO:tasks.rados.rados.0.smithi017.stdout:588: finishing write tid 4 to smithi017107884-2 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:588: oid 2 updating version 0 to 138 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:588: oid 2 version 138 is already newer than 138 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:588: oid 2 version 138 is already newer than 135 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 138 (ObjNum 286 snap 76 seq_num 286) dirty exists 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:588: left oid 2 (ObjNum 286 snap 76 seq_num 286) 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:588: done (2 left) 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:589: done (1 left) 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:590: done (0 left) 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:591: write (excl) oid 33 current snap is 76 2023-03-29T14:04:46.423 INFO:tasks.rados.rados.0.smithi017.stdout:591: seq_num 288 ranges {440416=776773,1686814=732100} 2023-03-29T14:04:46.429 INFO:tasks.rados.rados.0.smithi017.stdout:591: writing smithi017107884-33 from 440416 to 1217189 tid 1 2023-03-29T14:04:46.436 INFO:tasks.rados.rados.0.smithi017.stdout:591: writing smithi017107884-33 from 1686814 to 2418914 tid 2 2023-03-29T14:04:46.436 INFO:tasks.rados.rados.0.smithi017.stdout:592: read oid 27 snap -1 2023-03-29T14:04:46.436 INFO:tasks.rados.rados.0.smithi017.stdout:592: expect deleted 2023-03-29T14:04:46.436 INFO:tasks.rados.rados.0.smithi017.stdout:593: write oid 41 current snap is 76 2023-03-29T14:04:46.437 INFO:tasks.rados.rados.0.smithi017.stdout:593: seq_num 289 ranges {457223=568375,1461649=475758,2420186=437037} 2023-03-29T14:04:46.441 INFO:tasks.rados.rados.0.smithi017.stdout:593: writing smithi017107884-41 from 457223 to 1025598 tid 1 2023-03-29T14:04:46.446 INFO:tasks.rados.rados.0.smithi017.stdout:593: writing smithi017107884-41 from 1461649 to 1937407 tid 2 2023-03-29T14:04:46.451 INFO:tasks.rados.rados.0.smithi017.stdout:593: writing smithi017107884-41 from 2420186 to 2857223 tid 3 2023-03-29T14:04:46.451 INFO:tasks.rados.rados.0.smithi017.stdout:594: write (excl) oid 50 current snap is 76 2023-03-29T14:04:46.451 INFO:tasks.rados.rados.0.smithi017.stdout:594: seq_num 290 ranges {474030=643624,1520131=735769} 2023-03-29T14:04:46.457 INFO:tasks.rados.rados.0.smithi017.stdout:594: writing smithi017107884-50 from 474030 to 1117654 tid 1 2023-03-29T14:04:46.464 INFO:tasks.rados.rados.0.smithi017.stdout:594: writing smithi017107884-50 from 1520131 to 2255900 tid 2 2023-03-29T14:04:46.464 INFO:tasks.rados.rados.0.smithi017.stdout:595: write (excl) oid 20 current snap is 76 2023-03-29T14:04:46.464 INFO:tasks.rados.rados.0.smithi017.stdout:595: seq_num 291 ranges {490837=718873,1862260=479427,2875032=15805} 2023-03-29T14:04:46.471 INFO:tasks.rados.rados.0.smithi017.stdout:595: writing smithi017107884-20 from 490837 to 1209710 tid 1 2023-03-29T14:04:46.477 INFO:tasks.rados.rados.0.smithi017.stdout:595: writing smithi017107884-20 from 1862260 to 2341687 tid 2 2023-03-29T14:04:46.480 INFO:tasks.rados.rados.0.smithi017.stdout:595: writing smithi017107884-20 from 2875032 to 2890837 tid 3 2023-03-29T14:04:46.481 INFO:tasks.rados.rados.0.smithi017.stdout:596: read oid 35 snap 63 2023-03-29T14:04:46.481 INFO:tasks.rados.rados.0.smithi017.stdout:596: expect deleted 2023-03-29T14:04:46.481 INFO:tasks.rados.rados.0.smithi017.stdout:597: delete oid 23 current snap is 76 2023-03-29T14:04:46.510 INFO:tasks.rados.rados.0.smithi017.stdout:591: finishing write tid 1 to smithi017107884-33 2023-03-29T14:04:46.514 INFO:tasks.rados.rados.0.smithi017.stdout:593: finishing write tid 1 to smithi017107884-41 2023-03-29T14:04:46.550 INFO:tasks.rados.rados.0.smithi017.stdout:593: finishing write tid 2 to smithi017107884-41 2023-03-29T14:04:46.590 INFO:tasks.rados.rados.0.smithi017.stdout:591: finishing write tid 2 to smithi017107884-33 2023-03-29T14:04:46.596 INFO:tasks.rados.rados.0.smithi017.stdout:592: done (6 left) 2023-03-29T14:04:46.596 INFO:tasks.rados.rados.0.smithi017.stdout:597: done (5 left) 2023-03-29T14:04:46.596 INFO:tasks.rados.rados.0.smithi017.stdout:598: snap_remove snap 73 2023-03-29T14:04:46.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:46.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:46.964+0000 7fedcde3a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:46.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:46.968+0000 7fedcde3a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:47.117 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:47.117 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:47.117 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:47.221 INFO:tasks.rados.rados.0.smithi017.stdout:598: done (5 left) 2023-03-29T14:04:47.221 INFO:tasks.rados.rados.0.smithi017.stdout:599: snap_create 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: finishing write tid 3 to smithi017107884-41 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:591: finishing write tid 3 to smithi017107884-33 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: finishing write tid 4 to smithi017107884-41 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: finishing write tid 5 to smithi017107884-41 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: oid 41 updating version 0 to 99 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: oid 41 version 99 is already newer than 99 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: oid 41 version 99 is already newer than 99 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: oid 41 version 99 is already newer than 99 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 99 (ObjNum 289 snap 76 seq_num 289) dirty exists 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:593: left oid 41 (ObjNum 289 snap 76 seq_num 289) 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:591: finishing write tid 4 to smithi017107884-33 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:591: oid 33 updating version 0 to 71 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:591: oid 33 updating version 71 to 74 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:591: oid 33 version 74 is already newer than 74 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 74 (ObjNum 288 snap 76 seq_num 288) dirty exists 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:591: left oid 33 (ObjNum 288 snap 76 seq_num 288) 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:594: finishing write tid 1 to smithi017107884-50 2023-03-29T14:04:47.222 INFO:tasks.rados.rados.0.smithi017.stdout:595: finishing write tid 1 to smithi017107884-20 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:594: finishing write tid 2 to smithi017107884-50 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:594: finishing write tid 3 to smithi017107884-50 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:594: finishing write tid 4 to smithi017107884-50 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:594: oid 50 updating version 0 to 67 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:594: oid 50 version 67 is already newer than 64 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:594: oid 50 version 67 is already newer than 67 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 67 (ObjNum 290 snap 76 seq_num 290) dirty exists 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:594: left oid 50 (ObjNum 290 snap 76 seq_num 290) 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: finishing write tid 2 to smithi017107884-20 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: finishing write tid 3 to smithi017107884-20 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: finishing write tid 4 to smithi017107884-20 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: finishing write tid 5 to smithi017107884-20 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: oid 20 updating version 0 to 86 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: oid 20 updating version 86 to 90 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: oid 20 version 90 is already newer than 90 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: oid 20 version 90 is already newer than 90 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 90 (ObjNum 291 snap 76 seq_num 291) dirty exists 2023-03-29T14:04:47.223 INFO:tasks.rados.rados.0.smithi017.stdout:595: left oid 20 (ObjNum 291 snap 76 seq_num 291) 2023-03-29T14:04:47.250 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:47.247+0000 7fd941bd1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:47.253+0000 7fd941bd1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:47.456 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:47.459 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:47.470 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2023-03-29T14:04:47.219535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221488344,"num_objects":382,"num_object_clones":278,"num_object_copies":1146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1598,"num_read_kb":829448,"num_write":2204,"num_write_kb":560966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1045,"ondisk_log_size":1045,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":769004,"kb_used_data":720320,"kb_used_omap":0,"kb_used_meta":45952,"kb_avail":376718356,"statfs":{"total":386547056640,"available":385759596544,"internally_reserved":0,"allocated":737607680,"data_stored":887440694,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":47054848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12370150,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-4,"num_read":16,"num_read_kb":4490,"num_write":40,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022091"},"pg_stats":[{"pgid":"3.f","version":"151'35","reported_seq":197,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-03-29T14:04:40.255712+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:40.255712+0000","last_peered":"2023-03-29T14:04:40.255712+0000","last_clean":"2023-03-29T14:04:40.255712+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:40.255712+0000","last_undegraded":"2023-03-29T14:04:40.255712+0000","last_fullsized":"2023-03-29T14:04:40.255712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":90,"num_read_kb":60200,"num_write":68,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":153,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.351473+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:44.351473+0000","last_peered":"2023-03-29T14:04:44.351473+0000","last_clean":"2023-03-29T14:04:44.351473+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:44.351473+0000","last_undegraded":"2023-03-29T14:04:44.351473+0000","last_fullsized":"2023-03-29T14:04:44.351473+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"147'90","reported_seq":319,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.265115+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:44.265115+0000","last_peered":"2023-03-29T14:04:44.265115+0000","last_clean":"2023-03-29T14:04:44.265115+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:44.265115+0000","last_undegraded":"2023-03-29T14:04:44.265115+0000","last_fullsized":"2023-03-29T14:04:44.265115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17378504,"num_objects":30,"num_object_clones":20,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":104,"num_read_kb":56305,"num_write":182,"num_write_kb":54224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"145'83","reported_seq":308,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.299587+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:44.299587+0000","last_peered":"2023-03-29T14:04:44.299587+0000","last_clean":"2023-03-29T14:04:44.299587+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:44.299587+0000","last_undegraded":"2023-03-29T14:04:44.299587+0000","last_fullsized":"2023-03-29T14:04:44.299587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14185306,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":121,"num_read_kb":60885,"num_write":168,"num_write_kb":46564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"160'85","reported_seq":319,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:45.359089+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:45.359089+0000","last_peered":"2023-03-29T14:04:45.359089+0000","last_clean":"2023-03-29T14:04:45.359089+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:45.359089+0000","last_undegraded":"2023-03-29T14:04:45.359089+0000","last_fullsized":"2023-03-29T14:04:45.359089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20706488,"num_objects":34,"num_object_clones":26,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":71226,"num_write":174,"num_write_kb":51848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":121,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.293480+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:44.293480+0000","last_peered":"2023-03-29T14:04:44.293480+0000","last_clean":"2023-03-29T14:04:44.293480+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:44.293480+0000","last_undegraded":"2023-03-29T14:04:44.293480+0000","last_fullsized":"2023-03-29T14:04:44.293480+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"139'53","reported_seq":249,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.321040+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:44.321040+0000","last_peered":"2023-03-29T14:04:44.321040+0000","last_clean":"2023-03-29T14:04:44.321040+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:44.321040+0000","last_undegraded":"2023-03-29T14:04:44.321040+0000","last_fullsized":"2023-03-29T14:04:44.321040+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11697738,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":93,"num_read_kb":51652,"num_write":106,"num_write_kb":29126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"148'46","reported_seq":227,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.276620+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:44.276620+0000","last_peered":"2023-03-29T14:04:44.276620+0000","last_clean":"2023-03-29T14:04:44.276620+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:44.276620+0000","last_undegraded":"2023-03-29T14:04:44.276620+0000","last_fullsized":"2023-03-29T14:04:44.276620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11294370,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":34279,"num_write":100,"num_write_kb":28416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.300385+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:44.300385+0000","last_peered":"2023-03-29T14:04:44.300385+0000","last_clean":"2023-03-29T14:04:44.300385+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:44.300385+0000","last_undegraded":"2023-03-29T14:04:44.300385+0000","last_fullsized":"2023-03-29T14:04:44.300385+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'33","reported_seq":168,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-03-29T14:04:41.239163+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:41.239163+0000","last_peered":"2023-03-29T14:04:41.239163+0000","last_clean":"2023-03-29T14:04:41.239163+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:41.239163+0000","last_undegraded":"2023-03-29T14:04:41.239163+0000","last_fullsized":"2023-03-29T14:04:41.239163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":43,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.294607+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:44.294607+0000","last_peered":"2023-03-29T14:04:44.294607+0000","last_clean":"2023-03-29T14:04:44.294607+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:44.294607+0000","last_undegraded":"2023-03-29T14:04:44.294607+0000","last_fullsized":"2023-03-29T14:04:44.294607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"148'49","reported_seq":233,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:42.265416+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:42.265416+0000","last_peered":"2023-03-29T14:04:42.265416+0000","last_clean":"2023-03-29T14:04:42.265416+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:42.265416+0000","last_undegraded":"2023-03-29T14:04:42.265416+0000","last_fullsized":"2023-03-29T14:04:42.265416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12840614,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":40086,"num_write":102,"num_write_kb":28710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":104,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-03-29T14:04:40.275665+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:40.275665+0000","last_peered":"2023-03-29T14:04:40.275665+0000","last_clean":"2023-03-29T14:04:40.275665+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:40.275665+0000","last_undegraded":"2023-03-29T14:04:40.275665+0000","last_fullsized":"2023-03-29T14:04:40.275665+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"115'63","reported_seq":268,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:45.247002+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:45.247002+0000","last_peered":"2023-03-29T14:04:45.247002+0000","last_clean":"2023-03-29T14:04:45.247002+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:45.247002+0000","last_undegraded":"2023-03-29T14:04:45.247002+0000","last_fullsized":"2023-03-29T14:04:45.247002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622222,"num_objects":24,"num_object_clones":18,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":45588,"num_write":132,"num_write_kb":40486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.263989+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:44.263989+0000","last_peered":"2023-03-29T14:04:44.263989+0000","last_clean":"2023-03-29T14:04:44.263989+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:44.263989+0000","last_undegraded":"2023-03-29T14:04:44.263989+0000","last_fullsized":"2023-03-29T14:04:44.263989+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":349,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.318044+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:44.318044+0000","last_peered":"2023-03-29T14:04:44.318044+0000","last_clean":"2023-03-29T14:04:44.318044+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:44.318044+0000","last_undegraded":"2023-03-29T14:04:44.318044+0000","last_fullsized":"2023-03-29T14:04:44.318044+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"150'60","reported_seq":281,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.290675+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:44.290675+0000","last_peered":"2023-03-29T14:04:44.290675+0000","last_clean":"2023-03-29T14:04:44.290675+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:44.290675+0000","last_undegraded":"2023-03-29T14:04:44.290675+0000","last_fullsized":"2023-03-29T14:04:44.290675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15966650,"num_objects":24,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":134,"num_read_kb":84070,"num_write":124,"num_write_kb":35404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.301452+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:44.301452+0000","last_peered":"2023-03-29T14:04:44.301452+0000","last_clean":"2023-03-29T14:04:44.301452+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:44.301452+0000","last_undegraded":"2023-03-29T14:04:44.301452+0000","last_fullsized":"2023-03-29T14:04:44.301452+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"146'134","reported_seq":431,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.247720+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:44.247720+0000","last_peered":"2023-03-29T14:04:44.247720+0000","last_clean":"2023-03-29T14:04:44.247720+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:44.247720+0000","last_undegraded":"2023-03-29T14:04:44.247720+0000","last_fullsized":"2023-03-29T14:04:44.247720+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37614396,"num_objects":46,"num_object_clones":32,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":106525,"num_write":274,"num_write_kb":77992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:45.227813+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:45.227813+0000","last_peered":"2023-03-29T14:04:45.227813+0000","last_clean":"2023-03-29T14:04:45.227813+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:45.227813+0000","last_undegraded":"2023-03-29T14:04:45.227813+0000","last_fullsized":"2023-03-29T14:04:45.227813+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"145'43","reported_seq":223,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.321499+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:44.321499+0000","last_peered":"2023-03-29T14:04:44.321499+0000","last_clean":"2023-03-29T14:04:44.321499+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:44.321499+0000","last_undegraded":"2023-03-29T14:04:44.321499+0000","last_fullsized":"2023-03-29T14:04:44.321499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437114,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":70,"num_read_kb":44595,"num_write":90,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"147'94","reported_seq":328,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:45.239754+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:45.239754+0000","last_peered":"2023-03-29T14:04:45.239754+0000","last_clean":"2023-03-29T14:04:45.239754+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:45.239754+0000","last_undegraded":"2023-03-29T14:04:45.239754+0000","last_fullsized":"2023-03-29T14:04:45.239754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13681096,"num_objects":38,"num_object_clones":28,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":122,"num_read_kb":64144,"num_write":186,"num_write_kb":56676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:45.233275+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:45.233275+0000","last_peered":"2023-03-29T14:04:45.233275+0000","last_clean":"2023-03-29T14:04:45.233275+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:45.233275+0000","last_undegraded":"2023-03-29T14:04:45.233275+0000","last_fullsized":"2023-03-29T14:04:45.233275+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.279491+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:44.279491+0000","last_peered":"2023-03-29T14:04:44.279491+0000","last_clean":"2023-03-29T14:04:44.279491+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:44.279491+0000","last_undegraded":"2023-03-29T14:04:44.279491+0000","last_fullsized":"2023-03-29T14:04:44.279491+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"159'68","reported_seq":286,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-03-29T14:04:44.434585+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:44.434585+0000","last_peered":"2023-03-29T14:04:44.434585+0000","last_clean":"2023-03-29T14:04:44.434585+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:44.434585+0000","last_undegraded":"2023-03-29T14:04:44.434585+0000","last_fullsized":"2023-03-29T14:04:44.434585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24874426,"num_objects":26,"num_object_clones":18,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":67584,"num_write":140,"num_write_kb":39338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":220307608,"num_objects":378,"num_object_clones":278,"num_object_copies":1134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":1456,"num_read_kb":829328,"num_write":1930,"num_write_kb":557178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":948,"ondisk_log_size":948,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":201512,"kb_used_data":188252,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94170328,"statfs":{"total":96636764160,"available":96430415872,"internally_reserved":0,"allocated":192770048,"data_stored":231371673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":131320,"kb_used_data":123696,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94240520,"statfs":{"total":96636764160,"available":96502292480,"internally_reserved":0,"allocated":126664704,"data_stored":148676216,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705823,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":227928,"kb_used_data":213836,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94143912,"statfs":{"total":96636764160,"available":96403365888,"internally_reserved":0,"allocated":218968064,"data_stored":265123556,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705823,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":208244,"kb_used_data":194536,"kb_used_omap":0,"kb_used_meta":12800,"kb_avail":94163596,"statfs":{"total":96636764160,"available":96423522304,"internally_reserved":0,"allocated":199204864,"data_stored":242269249,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13107200},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:48.227 INFO:tasks.rados.rados.0.smithi017.stdout:591: done (5 left) 2023-03-29T14:04:48.227 INFO:tasks.rados.rados.0.smithi017.stdout:593: done (4 left) 2023-03-29T14:04:48.227 INFO:tasks.rados.rados.0.smithi017.stdout:594: done (3 left) 2023-03-29T14:04:48.227 INFO:tasks.rados.rados.0.smithi017.stdout:595: done (2 left) 2023-03-29T14:04:48.227 INFO:tasks.rados.rados.0.smithi017.stdout:596: done (1 left) 2023-03-29T14:04:48.227 INFO:tasks.rados.rados.0.smithi017.stdout:599: done (0 left) 2023-03-29T14:04:48.227 INFO:tasks.rados.rados.0.smithi017.stdout:600: snap_create 2023-03-29T14:04:49.038 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:49.184 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:49.183+0000 7f7a1cc71700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:49.187 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:49.187+0000 7f7a1cc71700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:49.230 INFO:tasks.rados.rados.0.smithi017.stdout:600: done (0 left) 2023-03-29T14:04:49.230 INFO:tasks.rados.rados.0.smithi017.stdout:601: write oid 29 current snap is 78 2023-03-29T14:04:49.230 INFO:tasks.rados.rados.0.smithi017.stdout:601: seq_num 293 ranges {524451=469371,1579224=483096,2529878=394573} 2023-03-29T14:04:49.235 INFO:tasks.rados.rados.0.smithi017.stdout:601: writing smithi017107884-29 from 524451 to 993822 tid 1 2023-03-29T14:04:49.241 INFO:tasks.rados.rados.0.smithi017.stdout:601: writing smithi017107884-29 from 1579224 to 2062320 tid 2 2023-03-29T14:04:49.246 INFO:tasks.rados.rados.0.smithi017.stdout:601: writing smithi017107884-29 from 2529878 to 2924451 tid 3 2023-03-29T14:04:49.246 INFO:tasks.rados.rados.0.smithi017.stdout:602: write (excl) oid 19 current snap is 78 2023-03-29T14:04:49.246 INFO:tasks.rados.rados.0.smithi017.stdout:602: seq_num 294 ranges {541258=544620,1637706=626754,2840948=100310} 2023-03-29T14:04:49.252 INFO:tasks.rados.rados.0.smithi017.stdout:602: writing smithi017107884-19 from 541258 to 1085878 tid 1 2023-03-29T14:04:49.258 INFO:tasks.rados.rados.0.smithi017.stdout:602: writing smithi017107884-19 from 1637706 to 2264460 tid 2 2023-03-29T14:04:49.262 INFO:tasks.rados.rados.0.smithi017.stdout:602: writing smithi017107884-19 from 2840948 to 2941258 tid 3 2023-03-29T14:04:49.262 INFO:tasks.rados.rados.0.smithi017.stdout:603: write (excl) oid 31 current snap is 78 2023-03-29T14:04:49.262 INFO:tasks.rados.rados.0.smithi017.stdout:603: seq_num 295 ranges {558065=619869,1579835=486765,2468371=489694} 2023-03-29T14:04:49.269 INFO:tasks.rados.rados.0.smithi017.stdout:603: writing smithi017107884-31 from 558065 to 1177934 tid 1 2023-03-29T14:04:49.274 INFO:tasks.rados.rados.0.smithi017.stdout:603: writing smithi017107884-31 from 1579835 to 2066600 tid 2 2023-03-29T14:04:49.279 INFO:tasks.rados.rados.0.smithi017.stdout:603: writing smithi017107884-31 from 2468371 to 2958065 tid 3 2023-03-29T14:04:49.279 INFO:tasks.rados.rados.0.smithi017.stdout:604: read oid 22 snap -1 2023-03-29T14:04:49.280 INFO:tasks.rados.rados.0.smithi017.stdout:604: expect (ObjNum 276 snap 72 seq_num 276) 2023-03-29T14:04:49.280 INFO:tasks.rados.rados.0.smithi017.stdout:605: read oid 12 snap 67 2023-03-29T14:04:49.280 INFO:tasks.rados.rados.0.smithi017.stdout:605: expect deleted 2023-03-29T14:04:49.280 INFO:tasks.rados.rados.0.smithi017.stdout:606: read oid 9 snap -1 2023-03-29T14:04:49.280 INFO:tasks.rados.rados.0.smithi017.stdout:606: expect (ObjNum 256 snap 70 seq_num 256) 2023-03-29T14:04:49.280 INFO:tasks.rados.rados.0.smithi017.stdout:607: write (excl) oid 24 current snap is 78 2023-03-29T14:04:49.280 INFO:tasks.rados.rados.0.smithi017.stdout:607: seq_num 296 ranges {574872=695118,2038317=630423} 2023-03-29T14:04:49.286 INFO:tasks.rados.rados.0.smithi017.stdout:607: writing smithi017107884-24 from 574872 to 1269990 tid 1 2023-03-29T14:04:49.294 INFO:tasks.rados.rados.0.smithi017.stdout:607: writing smithi017107884-24 from 2038317 to 2668740 tid 2 2023-03-29T14:04:49.294 INFO:tasks.rados.rados.0.smithi017.stdout:608: read oid 41 snap -1 2023-03-29T14:04:49.294 INFO:tasks.rados.rados.0.smithi017.stdout:608: expect (ObjNum 289 snap 76 seq_num 289) 2023-03-29T14:04:49.295 INFO:tasks.rados.rados.0.smithi017.stdout:609: read oid 17 snap -1 2023-03-29T14:04:49.295 INFO:tasks.rados.rados.0.smithi017.stdout:609: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:49.295 INFO:tasks.rados.rados.0.smithi017.stdout:610: delete oid 40 current snap is 78 2023-03-29T14:04:49.361 INFO:tasks.rados.rados.0.smithi017.stdout:604: expect (ObjNum 276 snap 72 seq_num 276) 2023-03-29T14:04:49.514 INFO:tasks.rados.rados.0.smithi017.stdout:606: expect (ObjNum 256 snap 70 seq_num 256) 2023-03-29T14:04:49.689 INFO:tasks.rados.rados.0.smithi017.stdout:601: finishing write tid 1 to smithi017107884-29 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:603: finishing write tid 1 to smithi017107884-31 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:604: done (9 left) 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:605: done (8 left) 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:606: done (7 left) 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:610: done (6 left) 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:611: read oid 15 snap 71 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:611: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:612: write (excl) oid 42 current snap is 78 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:601: finishing write tid 2 to smithi017107884-29 2023-03-29T14:04:49.690 INFO:tasks.rados.rados.0.smithi017.stdout:609: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:49.931 INFO:tasks.rados.rados.0.smithi017.stdout:612: seq_num 298 ranges {608486=561969,1871634=634092,2950640=57846} 2023-03-29T14:04:49.937 INFO:tasks.rados.rados.0.smithi017.stdout:612: writing smithi017107884-42 from 608486 to 1170455 tid 1 2023-03-29T14:04:49.944 INFO:tasks.rados.rados.0.smithi017.stdout:612: writing smithi017107884-42 from 1871634 to 2505726 tid 2 2023-03-29T14:04:49.947 INFO:tasks.rados.rados.0.smithi017.stdout:612: writing smithi017107884-42 from 2950640 to 3008486 tid 3 2023-03-29T14:04:49.947 INFO:tasks.rados.rados.0.smithi017.stdout:609: done (7 left) 2023-03-29T14:04:49.948 INFO:tasks.rados.rados.0.smithi017.stdout:613: delete oid 8 current snap is 78 2023-03-29T14:04:49.948 INFO:tasks.rados.rados.0.smithi017.stdout:603: finishing write tid 2 to smithi017107884-31 2023-03-29T14:04:49.948 INFO:tasks.rados.rados.0.smithi017.stdout:601: finishing write tid 3 to smithi017107884-29 2023-03-29T14:04:49.948 INFO:tasks.rados.rados.0.smithi017.stdout:601: finishing write tid 4 to smithi017107884-29 2023-03-29T14:04:49.948 INFO:tasks.rados.rados.0.smithi017.stdout:608: expect (ObjNum 289 snap 76 seq_num 289) 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:601: finishing write tid 5 to smithi017107884-29 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:601: oid 29 updating version 0 to 54 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:601: oid 29 version 54 is already newer than 54 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:601: oid 29 version 54 is already newer than 54 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:601: oid 29 version 54 is already newer than 54 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 54 (ObjNum 293 snap 78 seq_num 293) dirty exists 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:601: left oid 29 (ObjNum 293 snap 78 seq_num 293) 2023-03-29T14:04:50.131 INFO:tasks.rados.rados.0.smithi017.stdout:601: done (7 left) 2023-03-29T14:04:50.132 INFO:tasks.rados.rados.0.smithi017.stdout:608: done (6 left) 2023-03-29T14:04:50.132 INFO:tasks.rados.rados.0.smithi017.stdout:613: done (5 left) 2023-03-29T14:04:50.132 INFO:tasks.rados.rados.0.smithi017.stdout:614: write oid 37 current snap is 78 2023-03-29T14:04:50.132 INFO:tasks.rados.rados.0.smithi017.stdout:603: finishing write tid 3 to smithi017107884-31 2023-03-29T14:04:50.132 INFO:tasks.rados.rados.0.smithi017.stdout:603: finishing write tid 4 to smithi017107884-31 2023-03-29T14:04:50.132 INFO:tasks.rados.rados.0.smithi017.stdout:603: finishing write tid 5 to smithi017107884-31 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:603: oid 31 updating version 0 to 58 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:603: oid 31 version 58 is already newer than 58 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:603: oid 31 version 58 is already newer than 58 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:603: oid 31 version 58 is already newer than 55 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 58 (ObjNum 295 snap 78 seq_num 295) dirty exists 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:603: left oid 31 (ObjNum 295 snap 78 seq_num 295) 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:602: finishing write tid 1 to smithi017107884-19 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:607: finishing write tid 1 to smithi017107884-24 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:602: finishing write tid 2 to smithi017107884-19 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:602: finishing write tid 3 to smithi017107884-19 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:602: finishing write tid 4 to smithi017107884-19 2023-03-29T14:04:50.133 INFO:tasks.rados.rados.0.smithi017.stdout:614: seq_num 300 ranges {642100=712467,1872245=637761} 2023-03-29T14:04:50.140 INFO:tasks.rados.rados.0.smithi017.stdout:614: writing smithi017107884-37 from 642100 to 1354567 tid 1 2023-03-29T14:04:50.145 INFO:tasks.rados.rados.0.smithi017.stdout:614: writing smithi017107884-37 from 1872245 to 2510006 tid 2 2023-03-29T14:04:50.145 INFO:tasks.rados.rados.0.smithi017.stdout:603: done (5 left) 2023-03-29T14:04:50.146 INFO:tasks.rados.rados.0.smithi017.stdout:615: write oid 44 current snap is 78 2023-03-29T14:04:50.146 INFO:tasks.rados.rados.0.smithi017.stdout:602: finishing write tid 5 to smithi017107884-19 2023-03-29T14:04:50.146 INFO:tasks.rados.rados.0.smithi017.stdout:602: oid 19 updating version 0 to 79 2023-03-29T14:04:50.146 INFO:tasks.rados.rados.0.smithi017.stdout:602: oid 19 version 79 is already newer than 75 2023-03-29T14:04:50.146 INFO:tasks.rados.rados.0.smithi017.stdout:602: oid 19 version 79 is already newer than 79 2023-03-29T14:04:50.146 INFO:tasks.rados.rados.0.smithi017.stdout:602: oid 19 version 79 is already newer than 79 2023-03-29T14:04:50.147 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 79 (ObjNum 294 snap 78 seq_num 294) dirty exists 2023-03-29T14:04:50.147 INFO:tasks.rados.rados.0.smithi017.stdout:602: left oid 19 (ObjNum 294 snap 78 seq_num 294) 2023-03-29T14:04:50.147 INFO:tasks.rados.rados.0.smithi017.stdout:607: finishing write tid 2 to smithi017107884-24 2023-03-29T14:04:50.147 INFO:tasks.rados.rados.0.smithi017.stdout:615: seq_num 301 ranges {658907=787716,1930727=497772,3032909=25998} 2023-03-29T14:04:50.153 INFO:tasks.rados.rados.0.smithi017.stdout:615: writing smithi017107884-44 from 658907 to 1446623 tid 1 2023-03-29T14:04:50.158 INFO:tasks.rados.rados.0.smithi017.stdout:615: writing smithi017107884-44 from 1930727 to 2428499 tid 2 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:615: writing smithi017107884-44 from 3032909 to 3058907 tid 3 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:602: done (5 left) 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:616: delete oid 14 current snap is 78 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:607: finishing write tid 3 to smithi017107884-24 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:607: finishing write tid 4 to smithi017107884-24 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:607: oid 24 updating version 0 to 55 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:607: oid 24 updating version 55 to 58 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:607: oid 24 version 58 is already newer than 58 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 58 (ObjNum 296 snap 78 seq_num 296) dirty exists 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:607: left oid 24 (ObjNum 296 snap 78 seq_num 296) 2023-03-29T14:04:50.162 INFO:tasks.rados.rados.0.smithi017.stdout:611: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: finishing write tid 1 to smithi017107884-42 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: finishing write tid 2 to smithi017107884-42 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: finishing write tid 3 to smithi017107884-42 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: finishing write tid 4 to smithi017107884-42 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: finishing write tid 5 to smithi017107884-42 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: oid 42 updating version 0 to 86 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: oid 42 updating version 86 to 89 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: oid 42 version 89 is already newer than 89 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: oid 42 version 89 is already newer than 89 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 89 (ObjNum 298 snap 78 seq_num 298) dirty exists 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:612: left oid 42 (ObjNum 298 snap 78 seq_num 298) 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:607: done (5 left) 2023-03-29T14:04:50.293 INFO:tasks.rados.rados.0.smithi017.stdout:611: done (4 left) 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:612: done (3 left) 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:616: done (2 left) 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:617: write oid 7 current snap is 78 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:614: finishing write tid 1 to smithi017107884-37 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: finishing write tid 1 to smithi017107884-44 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:614: finishing write tid 2 to smithi017107884-37 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: finishing write tid 2 to smithi017107884-44 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: finishing write tid 3 to smithi017107884-44 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: finishing write tid 4 to smithi017107884-44 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: finishing write tid 5 to smithi017107884-44 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: oid 44 updating version 0 to 47 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: oid 44 version 47 is already newer than 47 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: oid 44 version 47 is already newer than 47 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:615: oid 44 version 47 is already newer than 47 2023-03-29T14:04:50.294 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 47 (ObjNum 301 snap 78 seq_num 301) dirty exists 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:615: left oid 44 (ObjNum 301 snap 78 seq_num 301) 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:614: finishing write tid 3 to smithi017107884-37 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:614: finishing write tid 4 to smithi017107884-37 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:614: oid 37 updating version 0 to 37 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:614: oid 37 version 37 is already newer than 37 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:614: oid 37 version 37 is already newer than 37 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 37 (ObjNum 300 snap 78 seq_num 300) dirty exists 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:614: left oid 37 (ObjNum 300 snap 78 seq_num 300) 2023-03-29T14:04:50.295 INFO:tasks.rados.rados.0.smithi017.stdout:617: seq_num 303 ranges {692521=538214,1931338=785088} 2023-03-29T14:04:50.299 INFO:tasks.rados.rados.0.smithi017.stdout:617: writing smithi017107884-7 from 692521 to 1230735 tid 1 2023-03-29T14:04:50.307 INFO:tasks.rados.rados.0.smithi017.stdout:617: writing smithi017107884-7 from 1931338 to 2716426 tid 2 2023-03-29T14:04:50.307 INFO:tasks.rados.rados.0.smithi017.stdout:614: done (2 left) 2023-03-29T14:04:50.307 INFO:tasks.rados.rados.0.smithi017.stdout:615: done (1 left) 2023-03-29T14:04:50.307 INFO:tasks.rados.rados.0.smithi017.stdout:618: read oid 2 snap -1 2023-03-29T14:04:50.307 INFO:tasks.rados.rados.0.smithi017.stdout:618: expect (ObjNum 286 snap 76 seq_num 286) 2023-03-29T14:04:50.307 INFO:tasks.rados.rados.0.smithi017.stdout:619: write (excl) oid 1 current snap is 78 2023-03-29T14:04:50.307 INFO:tasks.rados.rados.0.smithi017.stdout:619: seq_num 304 ranges {709328=613463,1989820=645099} 2023-03-29T14:04:50.314 INFO:tasks.rados.rados.0.smithi017.stdout:619: writing smithi017107884-1 from 709328 to 1322791 tid 1 2023-03-29T14:04:50.322 INFO:tasks.rados.rados.0.smithi017.stdout:619: writing smithi017107884-1 from 1989820 to 2634919 tid 2 2023-03-29T14:04:50.322 INFO:tasks.rados.rados.0.smithi017.stdout:620: read oid 11 snap 72 2023-03-29T14:04:50.322 INFO:tasks.rados.rados.0.smithi017.stdout:620: expect (ObjNum 269 snap 70 seq_num 269) 2023-03-29T14:04:50.322 INFO:tasks.rados.rados.0.smithi017.stdout:621: delete oid 6 current snap is 78 2023-03-29T14:04:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:617: finishing write tid 1 to smithi017107884-7 2023-03-29T14:04:50.372 INFO:tasks.rados.rados.0.smithi017.stdout:618: expect (ObjNum 286 snap 76 seq_num 286) 2023-03-29T14:04:50.524 INFO:tasks.rados.rados.0.smithi017.stdout:617: finishing write tid 2 to smithi017107884-7 2023-03-29T14:04:50.524 INFO:tasks.rados.rados.0.smithi017.stdout:617: finishing write tid 3 to smithi017107884-7 2023-03-29T14:04:50.524 INFO:tasks.rados.rados.0.smithi017.stdout:617: finishing write tid 4 to smithi017107884-7 2023-03-29T14:04:50.524 INFO:tasks.rados.rados.0.smithi017.stdout:617: oid 7 updating version 0 to 50 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:617: oid 7 version 50 is already newer than 50 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:617: oid 7 version 50 is already newer than 50 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 50 (ObjNum 303 snap 78 seq_num 303) dirty exists 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:617: left oid 7 (ObjNum 303 snap 78 seq_num 303) 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: finishing write tid 1 to smithi017107884-1 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: finishing write tid 2 to smithi017107884-1 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: finishing write tid 3 to smithi017107884-1 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: finishing write tid 4 to smithi017107884-1 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: oid 1 updating version 0 to 84 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: oid 1 version 84 is already newer than 81 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: oid 1 version 84 is already newer than 84 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 84 (ObjNum 304 snap 78 seq_num 304) dirty exists 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: left oid 1 (ObjNum 304 snap 78 seq_num 304) 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:617: done (4 left) 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:618: done (3 left) 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:619: done (2 left) 2023-03-29T14:04:50.525 INFO:tasks.rados.rados.0.smithi017.stdout:621: done (1 left) 2023-03-29T14:04:50.526 INFO:tasks.rados.rados.0.smithi017.stdout:622: write oid 10 current snap is 78 2023-03-29T14:04:50.526 INFO:tasks.rados.rados.0.smithi017.stdout:620: expect (ObjNum 269 snap 70 seq_num 269) 2023-03-29T14:04:50.659 INFO:tasks.rados.rados.0.smithi017.stdout:622: seq_num 306 ranges {742942=480314,1823137=648768} 2023-03-29T14:04:50.665 INFO:tasks.rados.rados.0.smithi017.stdout:622: writing smithi017107884-10 from 742942 to 1223256 tid 1 2023-03-29T14:04:50.673 INFO:tasks.rados.rados.0.smithi017.stdout:622: writing smithi017107884-10 from 1823137 to 2471905 tid 2 2023-03-29T14:04:50.673 INFO:tasks.rados.rados.0.smithi017.stdout:620: done (1 left) 2023-03-29T14:04:50.673 INFO:tasks.rados.rados.0.smithi017.stdout:623: read oid 40 snap -1 2023-03-29T14:04:50.674 INFO:tasks.rados.rados.0.smithi017.stdout:623: expect deleted 2023-03-29T14:04:50.674 INFO:tasks.rados.rados.0.smithi017.stdout:624: snap_create 2023-03-29T14:04:50.697 INFO:tasks.rados.rados.0.smithi017.stdout:622: finishing write tid 1 to smithi017107884-10 2023-03-29T14:04:50.745 INFO:tasks.rados.rados.0.smithi017.stdout:622: finishing write tid 2 to smithi017107884-10 2023-03-29T14:04:50.745 INFO:tasks.rados.rados.0.smithi017.stdout:622: finishing write tid 3 to smithi017107884-10 2023-03-29T14:04:50.746 INFO:tasks.rados.rados.0.smithi017.stdout:622: finishing write tid 4 to smithi017107884-10 2023-03-29T14:04:50.747 INFO:tasks.rados.rados.0.smithi017.stdout:622: oid 10 updating version 0 to 95 2023-03-29T14:04:50.747 INFO:tasks.rados.rados.0.smithi017.stdout:622: oid 10 version 95 is already newer than 95 2023-03-29T14:04:50.747 INFO:tasks.rados.rados.0.smithi017.stdout:622: oid 10 version 95 is already newer than 95 2023-03-29T14:04:50.747 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 95 (ObjNum 306 snap 78 seq_num 306) dirty exists 2023-03-29T14:04:50.747 INFO:tasks.rados.rados.0.smithi017.stdout:622: left oid 10 (ObjNum 306 snap 78 seq_num 306) 2023-03-29T14:04:51.237 INFO:tasks.rados.rados.0.smithi017.stdout:622: done (2 left) 2023-03-29T14:04:51.237 INFO:tasks.rados.rados.0.smithi017.stdout:623: done (1 left) 2023-03-29T14:04:51.237 INFO:tasks.rados.rados.0.smithi017.stdout:624: done (0 left) 2023-03-29T14:04:51.237 INFO:tasks.rados.rados.0.smithi017.stdout:625: delete oid 4 current snap is 79 2023-03-29T14:04:51.269 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:51.291 INFO:tasks.rados.rados.0.smithi017.stdout:625: done (0 left) 2023-03-29T14:04:51.291 INFO:tasks.rados.rados.0.smithi017.stdout:626: delete oid 1 current snap is 79 2023-03-29T14:04:51.366 INFO:tasks.rados.rados.0.smithi017.stdout:626: done (0 left) 2023-03-29T14:04:51.366 INFO:tasks.rados.rados.0.smithi017.stdout:627: read oid 41 snap 53 2023-03-29T14:04:51.367 INFO:tasks.rados.rados.0.smithi017.stdout:627: expect (ObjNum 113 snap 10 seq_num 113) 2023-03-29T14:04:51.367 INFO:tasks.rados.rados.0.smithi017.stdout:628: read oid 28 snap -1 2023-03-29T14:04:51.367 INFO:tasks.rados.rados.0.smithi017.stdout:628: expect (ObjNum 182 snap 34 seq_num 182) 2023-03-29T14:04:51.367 INFO:tasks.rados.rados.0.smithi017.stdout:629: snap_remove snap 63 2023-03-29T14:04:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:51.402+0000 7f7cc3605700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:51.407+0000 7f7cc3605700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:52.243 INFO:tasks.rados.rados.0.smithi017.stdout:629: done (2 left) 2023-03-29T14:04:52.243 INFO:tasks.rados.rados.0.smithi017.stdout:630: delete oid 4 current snap is 79 2023-03-29T14:04:52.244 INFO:tasks.rados.rados.0.smithi017.stdout:628: expect (ObjNum 182 snap 34 seq_num 182) 2023-03-29T14:04:52.435 INFO:tasks.rados.rados.0.smithi017.stdout:627: expect (ObjNum 113 snap 10 seq_num 113) 2023-03-29T14:04:52.477 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:52.478 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:52.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:52.613 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:52.610+0000 7f17d8ba8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:52.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:52.617+0000 7f17d8ba8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:52.644 INFO:tasks.rados.rados.0.smithi017.stdout:627: done (2 left) 2023-03-29T14:04:52.644 INFO:tasks.rados.rados.0.smithi017.stdout:628: done (1 left) 2023-03-29T14:04:52.644 INFO:tasks.rados.rados.0.smithi017.stdout:630: done (0 left) 2023-03-29T14:04:52.644 INFO:tasks.rados.rados.0.smithi017.stdout:631: write oid 44 current snap is 79 2023-03-29T14:04:52.644 INFO:tasks.rados.rados.0.smithi017.stdout:631: seq_num 310 ranges {410170=781310,1940712=656106,3163363=46807} 2023-03-29T14:04:52.650 INFO:tasks.rados.rados.0.smithi017.stdout:631: writing smithi017107884-44 from 410170 to 1191480 tid 1 2023-03-29T14:04:52.658 INFO:tasks.rados.rados.0.smithi017.stdout:631: writing smithi017107884-44 from 1940712 to 2596818 tid 2 2023-03-29T14:04:52.662 INFO:tasks.rados.rados.0.smithi017.stdout:631: writing smithi017107884-44 from 3163363 to 3210170 tid 3 2023-03-29T14:04:52.662 INFO:tasks.rados.rados.0.smithi017.stdout:632: delete oid 19 current snap is 79 2023-03-29T14:04:52.764 INFO:tasks.rados.rados.0.smithi017.stdout:632: done (1 left) 2023-03-29T14:04:52.764 INFO:tasks.rados.rados.0.smithi017.stdout:633: read oid 42 snap -1 2023-03-29T14:04:52.764 INFO:tasks.rados.rados.0.smithi017.stdout:633: expect (ObjNum 298 snap 78 seq_num 298) 2023-03-29T14:04:52.764 INFO:tasks.rados.rados.0.smithi017.stdout:634: read oid 13 snap -1 2023-03-29T14:04:52.764 INFO:tasks.rados.rados.0.smithi017.stdout:634: expect deleted 2023-03-29T14:04:52.764 INFO:tasks.rados.rados.0.smithi017.stdout:635: read oid 15 snap -1 2023-03-29T14:04:52.765 INFO:tasks.rados.rados.0.smithi017.stdout:635: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:52.765 INFO:tasks.rados.rados.0.smithi017.stdout:636: read oid 22 snap -1 2023-03-29T14:04:52.765 INFO:tasks.rados.rados.0.smithi017.stdout:636: expect (ObjNum 276 snap 72 seq_num 276) 2023-03-29T14:04:52.765 INFO:tasks.rados.rados.0.smithi017.stdout:637: write (excl) oid 32 current snap is 79 2023-03-29T14:04:52.765 INFO:tasks.rados.rados.0.smithi017.stdout:637: seq_num 312 ranges {443784=648161,1657676=659775,2818209=425575} 2023-03-29T14:04:52.770 INFO:tasks.rados.rados.0.smithi017.stdout:637: writing smithi017107884-32 from 443784 to 1091945 tid 1 2023-03-29T14:04:52.777 INFO:tasks.rados.rados.0.smithi017.stdout:637: writing smithi017107884-32 from 1657676 to 2317451 tid 2 2023-03-29T14:04:52.782 INFO:tasks.rados.rados.0.smithi017.stdout:637: writing smithi017107884-32 from 2818209 to 3243784 tid 3 2023-03-29T14:04:52.782 INFO:tasks.rados.rados.0.smithi017.stdout:638: delete oid 31 current snap is 79 2023-03-29T14:04:52.793 INFO:tasks.rados.rados.0.smithi017.stdout:631: finishing write tid 1 to smithi017107884-44 2023-03-29T14:04:52.805 INFO:tasks.rados.rados.0.smithi017.stdout:635: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:04:52.823 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:52.840 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-03-29T14:04:52.240981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":215572742,"num_objects":414,"num_object_clones":310,"num_object_copies":1242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1694,"num_read_kb":878939,"num_write":2338,"num_write_kb":600462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1121,"ondisk_log_size":1121,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":816960,"kb_used_data":766112,"kb_used_omap":0,"kb_used_meta":50816,"kb_avail":376670400,"statfs":{"total":386547056640,"available":385710489600,"internally_reserved":0,"allocated":784498688,"data_stored":945843912,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":52035584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11394750,"num_objects":32,"num_object_clones":32,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":95,"num_read_kb":49490,"num_write":126,"num_write_kb":36668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.113928"},"pg_stats":[{"pgid":"3.f","version":"159'38","reported_seq":208,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-03-29T14:04:47.213038+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:47.213038+0000","last_peered":"2023-03-29T14:04:47.213038+0000","last_clean":"2023-03-29T14:04:47.213038+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:47.213038+0000","last_undegraded":"2023-03-29T14:04:47.213038+0000","last_fullsized":"2023-03-29T14:04:47.213038+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10756612,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":60201,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":155,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:46.301299+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:46.301299+0000","last_peered":"2023-03-29T14:04:46.301299+0000","last_clean":"2023-03-29T14:04:46.301299+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:46.301299+0000","last_undegraded":"2023-03-29T14:04:46.301299+0000","last_fullsized":"2023-03-29T14:04:46.301299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"164'95","reported_seq":333,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:51.267587+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:51.267587+0000","last_peered":"2023-03-29T14:04:51.267587+0000","last_clean":"2023-03-29T14:04:51.267587+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:51.267587+0000","last_undegraded":"2023-03-29T14:04:51.267587+0000","last_fullsized":"2023-03-29T14:04:51.267587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756744,"num_objects":32,"num_object_clones":22,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":113,"num_read_kb":64575,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"164'89","reported_seq":325,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:50.048619+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:50.048619+0000","last_peered":"2023-03-29T14:04:50.048619+0000","last_clean":"2023-03-29T14:04:50.048619+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:50.048619+0000","last_undegraded":"2023-03-29T14:04:50.048619+0000","last_fullsized":"2023-03-29T14:04:50.048619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15193726,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":131,"num_read_kb":67226,"num_write":178,"num_write_kb":49016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"161'90","reported_seq":329,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:46.679938+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:46.679938+0000","last_peered":"2023-03-29T14:04:46.679938+0000","last_clean":"2023-03-29T14:04:46.679938+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:46.679938+0000","last_undegraded":"2023-03-29T14:04:46.679938+0000","last_fullsized":"2023-03-29T14:04:46.679938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20941786,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":71227,"num_write":184,"num_write_kb":54224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":124,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:49.343849+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:49.343849+0000","last_peered":"2023-03-29T14:04:49.343849+0000","last_clean":"2023-03-29T14:04:49.343849+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:49.343849+0000","last_undegraded":"2023-03-29T14:04:49.343849+0000","last_fullsized":"2023-03-29T14:04:49.343849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"164'60","reported_seq":263,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:50.181120+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:50.181120+0000","last_peered":"2023-03-29T14:04:50.181120+0000","last_clean":"2023-03-29T14:04:50.181120+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:50.181120+0000","last_undegraded":"2023-03-29T14:04:50.181120+0000","last_fullsized":"2023-03-29T14:04:50.181120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10286016,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":51653,"num_write":116,"num_write_kb":32248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"164'50","reported_seq":240,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:51.381251+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:51.381251+0000","last_peered":"2023-03-29T14:04:51.381251+0000","last_clean":"2023-03-29T14:04:51.381251+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:51.381251+0000","last_undegraded":"2023-03-29T14:04:51.381251+0000","last_fullsized":"2023-03-29T14:04:51.381251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12302790,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":41710,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":126,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:46.310540+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:46.310540+0000","last_peered":"2023-03-29T14:04:46.310540+0000","last_clean":"2023-03-29T14:04:46.310540+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:46.310540+0000","last_undegraded":"2023-03-29T14:04:46.310540+0000","last_fullsized":"2023-03-29T14:04:46.310540+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'33","reported_seq":172,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-03-29T14:04:47.214539+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:47.214539+0000","last_peered":"2023-03-29T14:04:47.214539+0000","last_clean":"2023-03-29T14:04:47.214539+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:47.214539+0000","last_undegraded":"2023-03-29T14:04:47.214539+0000","last_fullsized":"2023-03-29T14:04:47.214539+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":43,"num_read_kb":24200,"num_write":58,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":127,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:49.328394+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:49.328394+0000","last_peered":"2023-03-29T14:04:49.328394+0000","last_clean":"2023-03-29T14:04:49.328394+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:49.328394+0000","last_undegraded":"2023-03-29T14:04:49.328394+0000","last_fullsized":"2023-03-29T14:04:49.328394+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":251,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:49.588415+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:49.588415+0000","last_peered":"2023-03-29T14:04:49.588415+0000","last_clean":"2023-03-29T14:04:49.588415+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:49.588415+0000","last_undegraded":"2023-03-29T14:04:49.588415+0000","last_fullsized":"2023-03-29T14:04:49.588415+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":108,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-03-29T14:04:47.197418+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:47.197418+0000","last_peered":"2023-03-29T14:04:47.197418+0000","last_clean":"2023-03-29T14:04:47.197418+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:47.197418+0000","last_undegraded":"2023-03-29T14:04:47.197418+0000","last_fullsized":"2023-03-29T14:04:47.197418+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":276,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:46.676676+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:46.676676+0000","last_peered":"2023-03-29T14:04:46.676676+0000","last_clean":"2023-03-29T14:04:46.676676+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:46.676676+0000","last_undegraded":"2023-03-29T14:04:46.676676+0000","last_fullsized":"2023-03-29T14:04:46.676676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":127,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:49.353822+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:49.353822+0000","last_peered":"2023-03-29T14:04:49.353822+0000","last_clean":"2023-03-29T14:04:49.353822+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:49.353822+0000","last_undegraded":"2023-03-29T14:04:49.353822+0000","last_fullsized":"2023-03-29T14:04:49.353822+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":351,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:46.304272+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:46.304272+0000","last_peered":"2023-03-29T14:04:46.304272+0000","last_clean":"2023-03-29T14:04:46.304272+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:46.304272+0000","last_undegraded":"2023-03-29T14:04:46.304272+0000","last_fullsized":"2023-03-29T14:04:46.304272+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"164'62","reported_seq":286,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:50.328688+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:50.328688+0000","last_peered":"2023-03-29T14:04:50.328688+0000","last_clean":"2023-03-29T14:04:50.328688+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:50.328688+0000","last_undegraded":"2023-03-29T14:04:50.328688+0000","last_fullsized":"2023-03-29T14:04:50.328688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10722800,"num_objects":26,"num_object_clones":18,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":136,"num_read_kb":84070,"num_write":124,"num_write_kb":35404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":127,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:49.323119+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:49.323119+0000","last_peered":"2023-03-29T14:04:49.323119+0000","last_clean":"2023-03-29T14:04:49.323119+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:49.323119+0000","last_undegraded":"2023-03-29T14:04:49.323119+0000","last_fullsized":"2023-03-29T14:04:49.323119+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"161'140","reported_seq":450,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:50.441010+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:50.441010+0000","last_peered":"2023-03-29T14:04:50.441010+0000","last_clean":"2023-03-29T14:04:50.441010+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:50.441010+0000","last_undegraded":"2023-03-29T14:04:50.441010+0000","last_fullsized":"2023-03-29T14:04:50.441010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31463034,"num_objects":50,"num_object_clones":36,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":213,"num_read_kb":125527,"num_write":282,"num_write_kb":80640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":126,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:46.275801+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:46.275801+0000","last_peered":"2023-03-29T14:04:46.275801+0000","last_clean":"2023-03-29T14:04:46.275801+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:46.275801+0000","last_undegraded":"2023-03-29T14:04:46.275801+0000","last_fullsized":"2023-03-29T14:04:46.275801+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"164'47","reported_seq":237,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:50.181499+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:50.181499+0000","last_peered":"2023-03-29T14:04:50.181499+0000","last_clean":"2023-03-29T14:04:50.181499+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:50.181499+0000","last_undegraded":"2023-03-29T14:04:50.181499+0000","last_fullsized":"2023-03-29T14:04:50.181499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10554928,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":47062,"num_write":100,"num_write_kb":29210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"161'99","reported_seq":341,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:49.478392+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:49.478392+0000","last_peered":"2023-03-29T14:04:49.478392+0000","last_clean":"2023-03-29T14:04:49.478392+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:49.478392+0000","last_undegraded":"2023-03-29T14:04:49.478392+0000","last_fullsized":"2023-03-29T14:04:49.478392+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15328182,"num_objects":40,"num_object_clones":30,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":130,"num_read_kb":70115,"num_write":196,"num_write_kb":59572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:46.257202+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:46.257202+0000","last_peered":"2023-03-29T14:04:46.257202+0000","last_clean":"2023-03-29T14:04:46.257202+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:46.257202+0000","last_undegraded":"2023-03-29T14:04:46.257202+0000","last_fullsized":"2023-03-29T14:04:46.257202+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-03-29T14:04:49.334777+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:49.334777+0000","last_peered":"2023-03-29T14:04:49.334777+0000","last_clean":"2023-03-29T14:04:49.334777+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:49.334777+0000","last_undegraded":"2023-03-29T14:04:49.334777+0000","last_fullsized":"2023-03-29T14:04:49.334777+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"164'84","reported_seq":314,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-03-29T14:04:50.411090+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:50.411090+0000","last_peered":"2023-03-29T14:04:50.411090+0000","last_clean":"2023-03-29T14:04:50.411090+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:50.411090+0000","last_undegraded":"2023-03-29T14:04:50.411090+0000","last_fullsized":"2023-03-29T14:04:50.411090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17782004,"num_objects":34,"num_object_clones":26,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":123,"num_read_kb":67587,"num_write":166,"num_write_kb":47232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":214392006,"num_objects":410,"num_object_clones":310,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":24,"num_read":1552,"num_read_kb":878819,"num_write":2064,"num_write_kb":596674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1024,"ondisk_log_size":1024,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705829,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":212652,"kb_used_data":200100,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94159188,"statfs":{"total":96636764160,"available":96419008512,"internally_reserved":0,"allocated":204902400,"data_stored":245840152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":138744,"kb_used_data":126960,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94233096,"statfs":{"total":96636764160,"available":96494690304,"internally_reserved":0,"allocated":130007040,"data_stored":152475231,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705828,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":243168,"kb_used_data":229720,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94128672,"statfs":{"total":96636764160,"available":96387760128,"internally_reserved":0,"allocated":235233280,"data_stored":285760396,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705828,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":222396,"kb_used_data":209332,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94149444,"statfs":{"total":96636764160,"available":96409030656,"internally_reserved":0,"allocated":214355968,"data_stored":261768133,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:52.935 INFO:tasks.rados.rados.0.smithi017.stdout:631: finishing write tid 2 to smithi017107884-44 2023-03-29T14:04:52.935 INFO:tasks.rados.rados.0.smithi017.stdout:631: finishing write tid 3 to smithi017107884-44 2023-03-29T14:04:52.935 INFO:tasks.rados.rados.0.smithi017.stdout:631: finishing write tid 4 to smithi017107884-44 2023-03-29T14:04:52.935 INFO:tasks.rados.rados.0.smithi017.stdout:633: expect (ObjNum 298 snap 78 seq_num 298) 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:631: finishing write tid 5 to smithi017107884-44 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:631: oid 44 updating version 0 to 52 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:631: oid 44 version 52 is already newer than 51 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:631: oid 44 version 52 is already newer than 52 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:631: oid 44 version 52 is already newer than 52 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 52 (ObjNum 310 snap 79 seq_num 310) dirty exists 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:631: left oid 44 (ObjNum 310 snap 79 seq_num 310) 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:631: done (6 left) 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:633: done (5 left) 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:634: done (4 left) 2023-03-29T14:04:53.053 INFO:tasks.rados.rados.0.smithi017.stdout:635: done (3 left) 2023-03-29T14:04:53.054 INFO:tasks.rados.rados.0.smithi017.stdout:638: done (2 left) 2023-03-29T14:04:53.054 INFO:tasks.rados.rados.0.smithi017.stdout:639: read oid 48 snap 76 2023-03-29T14:04:53.054 INFO:tasks.rados.rados.0.smithi017.stdout:639: expect deleted 2023-03-29T14:04:53.054 INFO:tasks.rados.rados.0.smithi017.stdout:640: delete oid 49 current snap is 79 2023-03-29T14:04:53.054 INFO:tasks.rados.rados.0.smithi017.stdout:637: finishing write tid 1 to smithi017107884-32 2023-03-29T14:04:53.054 INFO:tasks.rados.rados.0.smithi017.stdout:636: expect (ObjNum 276 snap 72 seq_num 276) 2023-03-29T14:04:53.201 INFO:tasks.rados.rados.0.smithi017.stdout:637: finishing write tid 2 to smithi017107884-32 2023-03-29T14:04:53.201 INFO:tasks.rados.rados.0.smithi017.stdout:637: finishing write tid 3 to smithi017107884-32 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: finishing write tid 4 to smithi017107884-32 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: finishing write tid 5 to smithi017107884-32 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: oid 32 updating version 0 to 145 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: oid 32 version 145 is already newer than 145 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: oid 32 version 145 is already newer than 141 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: oid 32 version 145 is already newer than 145 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 145 (ObjNum 312 snap 79 seq_num 312) dirty exists 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: left oid 32 (ObjNum 312 snap 79 seq_num 312) 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:636: done (3 left) 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:637: done (2 left) 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:640: done (1 left) 2023-03-29T14:04:53.202 INFO:tasks.rados.rados.0.smithi017.stdout:641: snap_remove snap 72 2023-03-29T14:04:53.480 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:53.613 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:53.612+0000 7f9d44aa3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:53.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:53.616+0000 7f9d44aa3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:54.201 INFO:tasks.rados.rados.0.smithi017.stdout:641: done (1 left) 2023-03-29T14:04:54.202 INFO:tasks.rados.rados.0.smithi017.stdout:642: snap_remove snap 77 2023-03-29T14:04:55.203 INFO:tasks.rados.rados.0.smithi017.stdout:642: done (1 left) 2023-03-29T14:04:55.203 INFO:tasks.rados.rados.0.smithi017.stdout:643: read oid 5 snap -1 2023-03-29T14:04:55.203 INFO:tasks.rados.rados.0.smithi017.stdout:643: expect (ObjNum 261 snap 70 seq_num 261) 2023-03-29T14:04:55.203 INFO:tasks.rados.rados.0.smithi017.stdout:644: write oid 6 current snap is 79 2023-03-29T14:04:55.203 INFO:tasks.rados.rados.0.smithi017.stdout:644: seq_num 315 ranges {494205=473908,1433122=407102,2384125=462684} 2023-03-29T14:04:55.208 INFO:tasks.rados.rados.0.smithi017.stdout:644: writing smithi017107884-6 from 494205 to 968113 tid 1 2023-03-29T14:04:55.213 INFO:tasks.rados.rados.0.smithi017.stdout:644: writing smithi017107884-6 from 1433122 to 1840224 tid 2 2023-03-29T14:04:55.218 INFO:tasks.rados.rados.0.smithi017.stdout:644: writing smithi017107884-6 from 2384125 to 2846809 tid 3 2023-03-29T14:04:55.219 INFO:tasks.rados.rados.0.smithi017.stdout:645: delete oid 40 current snap is 79 2023-03-29T14:04:55.264 INFO:tasks.rados.rados.0.smithi017.stdout:643: expect (ObjNum 261 snap 70 seq_num 261) 2023-03-29T14:04:55.389 INFO:tasks.rados.rados.0.smithi017.stdout:644: finishing write tid 1 to smithi017107884-6 2023-03-29T14:04:55.389 INFO:tasks.rados.rados.0.smithi017.stdout:644: finishing write tid 2 to smithi017107884-6 2023-03-29T14:04:55.389 INFO:tasks.rados.rados.0.smithi017.stdout:644: finishing write tid 3 to smithi017107884-6 2023-03-29T14:04:55.389 INFO:tasks.rados.rados.0.smithi017.stdout:644: finishing write tid 4 to smithi017107884-6 2023-03-29T14:04:55.389 INFO:tasks.rados.rados.0.smithi017.stdout:644: finishing write tid 5 to smithi017107884-6 2023-03-29T14:04:55.389 INFO:tasks.rados.rados.0.smithi017.stdout:644: oid 6 updating version 0 to 66 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:644: oid 6 version 66 is already newer than 66 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:644: oid 6 version 66 is already newer than 64 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:644: oid 6 version 66 is already newer than 65 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 66 (ObjNum 315 snap 79 seq_num 315) dirty exists 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:644: left oid 6 (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:639: done (3 left) 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:643: done (2 left) 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:644: done (1 left) 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:645: done (0 left) 2023-03-29T14:04:55.390 INFO:tasks.rados.rados.0.smithi017.stdout:646: snap_remove snap 78 2023-03-29T14:04:55.687 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:04:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:55.817+0000 7f7f4eac7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:55.821 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:55.821+0000 7f7f4eac7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:56.250 INFO:tasks.rados.rados.0.smithi017.stdout:646: done (0 left) 2023-03-29T14:04:56.250 INFO:tasks.rados.rados.0.smithi017.stdout:647: write oid 19 current snap is 79 2023-03-29T14:04:56.250 INFO:tasks.rados.rados.0.smithi017.stdout:647: seq_num 317 ranges {527819=624406,1833733=410771,2722618=601581} 2023-03-29T14:04:56.257 INFO:tasks.rados.rados.0.smithi017.stdout:647: writing smithi017107884-19 from 527819 to 1152225 tid 1 2023-03-29T14:04:56.262 INFO:tasks.rados.rados.0.smithi017.stdout:647: writing smithi017107884-19 from 1833733 to 2244504 tid 2 2023-03-29T14:04:56.269 INFO:tasks.rados.rados.0.smithi017.stdout:647: writing smithi017107884-19 from 2722618 to 3324199 tid 3 2023-03-29T14:04:56.269 INFO:tasks.rados.rados.0.smithi017.stdout:648: delete oid 25 current snap is 79 2023-03-29T14:04:56.354 INFO:tasks.rados.rados.0.smithi017.stdout:648: done (1 left) 2023-03-29T14:04:56.354 INFO:tasks.rados.rados.0.smithi017.stdout:649: read oid 17 snap -1 2023-03-29T14:04:56.354 INFO:tasks.rados.rados.0.smithi017.stdout:649: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:56.354 INFO:tasks.rados.rados.0.smithi017.stdout:650: delete oid 13 current snap is 79 2023-03-29T14:04:56.378 INFO:tasks.rados.rados.0.smithi017.stdout:647: finishing write tid 1 to smithi017107884-19 2023-03-29T14:04:56.380 INFO:tasks.rados.rados.0.smithi017.stdout:649: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:56.625 INFO:tasks.rados.rados.0.smithi017.stdout:647: finishing write tid 2 to smithi017107884-19 2023-03-29T14:04:56.625 INFO:tasks.rados.rados.0.smithi017.stdout:647: finishing write tid 3 to smithi017107884-19 2023-03-29T14:04:56.625 INFO:tasks.rados.rados.0.smithi017.stdout:647: finishing write tid 4 to smithi017107884-19 2023-03-29T14:04:56.625 INFO:tasks.rados.rados.0.smithi017.stdout:647: finishing write tid 5 to smithi017107884-19 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:647: oid 19 updating version 0 to 92 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:647: oid 19 version 92 is already newer than 92 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:647: oid 19 version 92 is already newer than 91 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:647: oid 19 version 92 is already newer than 92 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 92 (ObjNum 317 snap 79 seq_num 317) dirty exists 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:647: left oid 19 (ObjNum 317 snap 79 seq_num 317) 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:647: done (2 left) 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:649: done (1 left) 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:650: done (0 left) 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:651: read oid 14 snap -1 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:651: expect deleted 2023-03-29T14:04:56.626 INFO:tasks.rados.rados.0.smithi017.stdout:652: delete oid 7 current snap is 79 2023-03-29T14:04:56.682 INFO:tasks.rados.rados.0.smithi017.stdout:651: done (1 left) 2023-03-29T14:04:56.682 INFO:tasks.rados.rados.0.smithi017.stdout:652: done (0 left) 2023-03-29T14:04:56.682 INFO:tasks.rados.rados.0.smithi017.stdout:653: read oid 32 snap -1 2023-03-29T14:04:56.682 INFO:tasks.rados.rados.0.smithi017.stdout:653: expect (ObjNum 312 snap 79 seq_num 312) 2023-03-29T14:04:56.682 INFO:tasks.rados.rados.0.smithi017.stdout:654: snap_remove snap 67 2023-03-29T14:04:57.254 INFO:tasks.rados.rados.0.smithi017.stdout:654: done (1 left) 2023-03-29T14:04:57.254 INFO:tasks.rados.rados.0.smithi017.stdout:655: read oid 22 snap -1 2023-03-29T14:04:57.254 INFO:tasks.rados.rados.0.smithi017.stdout:655: expect (ObjNum 276 snap 72 seq_num 276) 2023-03-29T14:04:57.254 INFO:tasks.rados.rados.0.smithi017.stdout:656: write oid 9 current snap is 79 2023-03-29T14:04:57.254 INFO:tasks.rados.rados.0.smithi017.stdout:656: seq_num 321 ranges {595047=641755,1667661=418109,2548663=763022} 2023-03-29T14:04:57.261 INFO:tasks.rados.rados.0.smithi017.stdout:656: writing smithi017107884-9 from 595047 to 1236802 tid 1 2023-03-29T14:04:57.266 INFO:tasks.rados.rados.0.smithi017.stdout:656: writing smithi017107884-9 from 1667661 to 2085770 tid 2 2023-03-29T14:04:57.273 INFO:tasks.rados.rados.0.smithi017.stdout:656: writing smithi017107884-9 from 2548663 to 3311685 tid 3 2023-03-29T14:04:57.273 INFO:tasks.rados.rados.0.smithi017.stdout:657: read oid 25 snap -1 2023-03-29T14:04:57.273 INFO:tasks.rados.rados.0.smithi017.stdout:657: expect deleted 2023-03-29T14:04:57.273 INFO:tasks.rados.rados.0.smithi017.stdout:658: read oid 16 snap -1 2023-03-29T14:04:57.273 INFO:tasks.rados.rados.0.smithi017.stdout:658: expect (ObjNum 282 snap 75 seq_num 282) 2023-03-29T14:04:57.273 INFO:tasks.rados.rados.0.smithi017.stdout:659: write (excl) oid 41 current snap is 79 2023-03-29T14:04:57.273 INFO:tasks.rados.rados.0.smithi017.stdout:659: seq_num 322 ranges {611854=717004,2126143=561767,3259733=152121} 2023-03-29T14:04:57.280 INFO:tasks.rados.rados.0.smithi017.stdout:659: writing smithi017107884-41 from 611854 to 1328858 tid 1 2023-03-29T14:04:57.288 INFO:tasks.rados.rados.0.smithi017.stdout:659: writing smithi017107884-41 from 2126143 to 2687910 tid 2 2023-03-29T14:04:57.292 INFO:tasks.rados.rados.0.smithi017.stdout:659: writing smithi017107884-41 from 3259733 to 3411854 tid 3 2023-03-29T14:04:57.293 INFO:tasks.rados.rados.0.smithi017.stdout:660: read oid 3 snap -1 2023-03-29T14:04:57.293 INFO:tasks.rados.rados.0.smithi017.stdout:660: expect (ObjNum 270 snap 70 seq_num 270) 2023-03-29T14:04:57.293 INFO:tasks.rados.rados.0.smithi017.stdout:661: write (excl) oid 12 current snap is 79 2023-03-29T14:04:57.293 INFO:tasks.rados.rados.0.smithi017.stdout:661: seq_num 323 ranges {628661=792253,2184625=421778,3403509=25152} 2023-03-29T14:04:57.300 INFO:tasks.rados.rados.0.smithi017.stdout:661: writing smithi017107884-12 from 628661 to 1420914 tid 1 2023-03-29T14:04:57.307 INFO:tasks.rados.rados.0.smithi017.stdout:661: writing smithi017107884-12 from 2184625 to 2606403 tid 2 2023-03-29T14:04:57.311 INFO:tasks.rados.rados.0.smithi017.stdout:661: writing smithi017107884-12 from 3403509 to 3428661 tid 3 2023-03-29T14:04:57.311 INFO:tasks.rados.rados.0.smithi017.stdout:662: snap_remove snap 76 2023-03-29T14:04:57.845 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:04:57.846 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:04:57.846 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:04:57.896 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:04:57.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:57.980+0000 7f0d9e4d4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:57.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:57.986+0000 7f0d9e4d4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:58.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:58.027+0000 7fa781b65700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:04:58.032+0000 7fa781b65700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:04:58.212 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:04:58.214 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:04:58.226 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2023-03-29T14:04:58.129118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203303434,"num_objects":428,"num_object_clones":324,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":1749,"num_read_kb":923431,"num_write":2376,"num_write_kb":612014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1143,"ondisk_log_size":1143,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":847992,"kb_used_data":796504,"kb_used_omap":0,"kb_used_meta":51456,"kb_avail":376639368,"statfs":{"total":386547056640,"available":385678712832,"internally_reserved":0,"allocated":815620096,"data_stored":984248884,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":52690944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18353508,"num_objects":14,"num_object_clones":14,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":47,"num_read_kb":39090,"num_write":30,"num_write_kb":8914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000965"},"pg_stats":[{"pgid":"3.f","version":"159'38","reported_seq":218,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-03-29T14:04:52.867097+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:04:52.867097+0000","last_peered":"2023-03-29T14:04:52.867097+0000","last_clean":"2023-03-29T14:04:52.867097+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:04:52.867097+0000","last_undegraded":"2023-03-29T14:04:52.867097+0000","last_fullsized":"2023-03-29T14:04:52.867097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10756612,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":71003,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":159,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.223595+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:54.223595+0000","last_peered":"2023-03-29T14:04:54.223595+0000","last_clean":"2023-03-29T14:04:54.223595+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:54.223595+0000","last_undegraded":"2023-03-29T14:04:54.223595+0000","last_fullsized":"2023-03-29T14:04:54.223595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"164'95","reported_seq":340,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:56.365266+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:56.365266+0000","last_peered":"2023-03-29T14:04:56.365266+0000","last_clean":"2023-03-29T14:04:56.365266+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:56.365266+0000","last_undegraded":"2023-03-29T14:04:56.365266+0000","last_fullsized":"2023-03-29T14:04:56.365266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756744,"num_objects":32,"num_object_clones":22,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":120,"num_read_kb":75360,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"164'89","reported_seq":334,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:56.279631+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:04:56.279631+0000","last_peered":"2023-03-29T14:04:56.279631+0000","last_clean":"2023-03-29T14:04:56.279631+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:04:56.279631+0000","last_undegraded":"2023-03-29T14:04:56.279631+0000","last_fullsized":"2023-03-29T14:04:56.279631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10622156,"num_objects":42,"num_object_clones":34,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":138,"num_read_kb":70901,"num_write":178,"num_write_kb":49016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"161'90","reported_seq":333,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.224746+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:54.224746+0000","last_peered":"2023-03-29T14:04:54.224746+0000","last_clean":"2023-03-29T14:04:54.224746+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:54.224746+0000","last_undegraded":"2023-03-29T14:04:54.224746+0000","last_fullsized":"2023-03-29T14:04:54.224746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20941786,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":71227,"num_write":184,"num_write_kb":54224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":127,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.230826+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:55.230826+0000","last_peered":"2023-03-29T14:04:55.230826+0000","last_clean":"2023-03-29T14:04:55.230826+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:55.230826+0000","last_undegraded":"2023-03-29T14:04:55.230826+0000","last_fullsized":"2023-03-29T14:04:55.230826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"166'62","reported_seq":268,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.224779+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:55.224779+0000","last_peered":"2023-03-29T14:04:55.224779+0000","last_clean":"2023-03-29T14:04:55.224779+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:55.224779+0000","last_undegraded":"2023-03-29T14:04:55.224779+0000","last_fullsized":"2023-03-29T14:04:55.224779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4369886,"num_objects":28,"num_object_clones":22,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":98,"num_read_kb":51653,"num_write":116,"num_write_kb":32248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"164'50","reported_seq":243,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.233967+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:55.233967+0000","last_peered":"2023-03-29T14:04:55.233967+0000","last_clean":"2023-03-29T14:04:55.233967+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:55.233967+0000","last_undegraded":"2023-03-29T14:04:55.233967+0000","last_fullsized":"2023-03-29T14:04:55.233967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12302790,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":41710,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.221226+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:54.221226+0000","last_peered":"2023-03-29T14:04:54.221226+0000","last_clean":"2023-03-29T14:04:54.221226+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:54.221226+0000","last_undegraded":"2023-03-29T14:04:54.221226+0000","last_fullsized":"2023-03-29T14:04:54.221226+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":183,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-03-29T14:04:51.276896+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:51.276896+0000","last_peered":"2023-03-29T14:04:51.276896+0000","last_clean":"2023-03-29T14:04:51.276896+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:51.276896+0000","last_undegraded":"2023-03-29T14:04:51.276896+0000","last_fullsized":"2023-03-29T14:04:51.276896+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.222331+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:55.222331+0000","last_peered":"2023-03-29T14:04:55.222331+0000","last_clean":"2023-03-29T14:04:55.222331+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:55.222331+0000","last_undegraded":"2023-03-29T14:04:55.222331+0000","last_fullsized":"2023-03-29T14:04:55.222331+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":253,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:51.304336+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:51.304336+0000","last_peered":"2023-03-29T14:04:51.304336+0000","last_clean":"2023-03-29T14:04:51.304336+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:51.304336+0000","last_undegraded":"2023-03-29T14:04:51.304336+0000","last_fullsized":"2023-03-29T14:04:51.304336+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-03-29T14:04:51.266945+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:51.266945+0000","last_peered":"2023-03-29T14:04:51.266945+0000","last_clean":"2023-03-29T14:04:51.266945+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:51.266945+0000","last_undegraded":"2023-03-29T14:04:51.266945+0000","last_fullsized":"2023-03-29T14:04:51.266945+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":280,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.219976+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:54.219976+0000","last_peered":"2023-03-29T14:04:54.219976+0000","last_clean":"2023-03-29T14:04:54.219976+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:54.219976+0000","last_undegraded":"2023-03-29T14:04:54.219976+0000","last_fullsized":"2023-03-29T14:04:54.219976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":130,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.227842+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:55.227842+0000","last_peered":"2023-03-29T14:04:55.227842+0000","last_clean":"2023-03-29T14:04:55.227842+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:55.227842+0000","last_undegraded":"2023-03-29T14:04:55.227842+0000","last_fullsized":"2023-03-29T14:04:55.227842+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":355,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.222984+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:54.222984+0000","last_peered":"2023-03-29T14:04:54.222984+0000","last_clean":"2023-03-29T14:04:54.222984+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:54.222984+0000","last_undegraded":"2023-03-29T14:04:54.222984+0000","last_fullsized":"2023-03-29T14:04:54.222984+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"168'66","reported_seq":298,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.296823+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:55.296823+0000","last_peered":"2023-03-29T14:04:55.296823+0000","last_clean":"2023-03-29T14:04:55.296823+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:55.296823+0000","last_undegraded":"2023-03-29T14:04:55.296823+0000","last_fullsized":"2023-03-29T14:04:55.296823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17311210,"num_objects":26,"num_object_clones":18,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":137,"num_read_kb":84071,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.237078+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:55.237078+0000","last_peered":"2023-03-29T14:04:55.237078+0000","last_clean":"2023-03-29T14:04:55.237078+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:55.237078+0000","last_undegraded":"2023-03-29T14:04:55.237078+0000","last_fullsized":"2023-03-29T14:04:55.237078+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"166'145","reported_seq":466,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:53.056154+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:53.056154+0000","last_peered":"2023-03-29T14:04:53.056154+0000","last_clean":"2023-03-29T14:04:53.056154+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:53.056154+0000","last_undegraded":"2023-03-29T14:04:53.056154+0000","last_fullsized":"2023-03-29T14:04:53.056154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28807462,"num_objects":54,"num_object_clones":40,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":223,"num_read_kb":131703,"num_write":292,"num_write_kb":84028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.232254+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:54.232254+0000","last_peered":"2023-03-29T14:04:54.232254+0000","last_clean":"2023-03-29T14:04:54.232254+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:54.232254+0000","last_undegraded":"2023-03-29T14:04:54.232254+0000","last_fullsized":"2023-03-29T14:04:54.232254+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":250,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.235578+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:54.235578+0000","last_peered":"2023-03-29T14:04:54.235578+0000","last_clean":"2023-03-29T14:04:54.235578+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:54.235578+0000","last_undegraded":"2023-03-29T14:04:54.235578+0000","last_fullsized":"2023-03-29T14:04:54.235578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"161'99","reported_seq":351,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.260108+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:55.260108+0000","last_peered":"2023-03-29T14:04:55.260108+0000","last_clean":"2023-03-29T14:04:55.260108+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:55.260108+0000","last_undegraded":"2023-03-29T14:04:55.260108+0000","last_fullsized":"2023-03-29T14:04:55.260108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15328182,"num_objects":40,"num_object_clones":30,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":144,"num_read_kb":83166,"num_write":196,"num_write_kb":59572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":130,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-03-29T14:04:54.212953+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:54.212953+0000","last_peered":"2023-03-29T14:04:54.212953+0000","last_clean":"2023-03-29T14:04:54.212953+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:54.212953+0000","last_undegraded":"2023-03-29T14:04:54.212953+0000","last_fullsized":"2023-03-29T14:04:54.212953+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:55.237665+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:55.237665+0000","last_peered":"2023-03-29T14:04:55.237665+0000","last_clean":"2023-03-29T14:04:55.237665+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:55.237665+0000","last_undegraded":"2023-03-29T14:04:55.237665+0000","last_fullsized":"2023-03-29T14:04:55.237665+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"165'86","reported_seq":320,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-03-29T14:04:52.712931+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:52.712931+0000","last_peered":"2023-03-29T14:04:52.712931+0000","last_clean":"2023-03-29T14:04:52.712931+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:52.712931+0000","last_undegraded":"2023-03-29T14:04:52.712931+0000","last_fullsized":"2023-03-29T14:04:52.712931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5680832,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":123,"num_read_kb":67587,"num_write":166,"num_write_kb":47232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":202122698,"num_objects":424,"num_object_clones":324,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":1607,"num_read_kb":923311,"num_write":2102,"num_write_kb":608226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1046,"ondisk_log_size":1046,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":217280,"kb_used_data":204600,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94154560,"statfs":{"total":96636764160,"available":96414269440,"internally_reserved":0,"allocated":209510400,"data_stored":251037215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":154240,"kb_used_data":142200,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94217600,"statfs":{"total":96636764160,"available":96478822400,"internally_reserved":0,"allocated":145612800,"data_stored":171569880,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705833,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":248804,"kb_used_data":235228,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94123036,"statfs":{"total":96636764160,"available":96381988864,"internally_reserved":0,"allocated":240873472,"data_stored":293173242,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705833,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":227668,"kb_used_data":214476,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94144172,"statfs":{"total":96636764160,"available":96403632128,"internally_reserved":0,"allocated":219623424,"data_stored":268468547,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:04:58.259 INFO:tasks.rados.rados.0.smithi017.stdout:662: done (8 left) 2023-03-29T14:04:58.259 INFO:tasks.rados.rados.0.smithi017.stdout:663: delete oid 18 current snap is 79 2023-03-29T14:04:58.259 INFO:tasks.rados.rados.0.smithi017.stdout:653: expect (ObjNum 312 snap 79 seq_num 312) 2023-03-29T14:04:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:660: expect (ObjNum 270 snap 70 seq_num 270) 2023-03-29T14:04:58.657 INFO:tasks.rados.rados.0.smithi017.stdout:659: finishing write tid 1 to smithi017107884-41 2023-03-29T14:04:58.657 INFO:tasks.rados.rados.0.smithi017.stdout:656: finishing write tid 1 to smithi017107884-9 2023-03-29T14:04:58.658 INFO:tasks.rados.rados.0.smithi017.stdout:656: finishing write tid 2 to smithi017107884-9 2023-03-29T14:04:58.658 INFO:tasks.rados.rados.0.smithi017.stdout:655: expect (ObjNum 276 snap 72 seq_num 276) 2023-03-29T14:04:58.805 INFO:tasks.rados.rados.0.smithi017.stdout:656: finishing write tid 3 to smithi017107884-9 2023-03-29T14:04:58.805 INFO:tasks.rados.rados.0.smithi017.stdout:658: expect (ObjNum 282 snap 75 seq_num 282) 2023-03-29T14:04:58.910 INFO:tasks.rados.rados.0.smithi017.stdout:656: finishing write tid 4 to smithi017107884-9 2023-03-29T14:04:58.910 INFO:tasks.rados.rados.0.smithi017.stdout:656: finishing write tid 5 to smithi017107884-9 2023-03-29T14:04:58.910 INFO:tasks.rados.rados.0.smithi017.stdout:656: oid 9 updating version 0 to 97 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:656: oid 9 version 97 is already newer than 97 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:656: oid 9 version 97 is already newer than 97 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:656: oid 9 version 97 is already newer than 97 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 97 (ObjNum 321 snap 79 seq_num 321) dirty exists 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:656: left oid 9 (ObjNum 321 snap 79 seq_num 321) 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: finishing write tid 2 to smithi017107884-41 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: finishing write tid 3 to smithi017107884-41 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: finishing write tid 4 to smithi017107884-41 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: finishing write tid 5 to smithi017107884-41 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: oid 41 updating version 0 to 101 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: oid 41 updating version 101 to 105 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: oid 41 version 105 is already newer than 105 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: oid 41 version 105 is already newer than 105 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 105 (ObjNum 322 snap 79 seq_num 322) dirty exists 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:659: left oid 41 (ObjNum 322 snap 79 seq_num 322) 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:661: finishing write tid 1 to smithi017107884-12 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:661: finishing write tid 2 to smithi017107884-12 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:661: finishing write tid 3 to smithi017107884-12 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:661: finishing write tid 4 to smithi017107884-12 2023-03-29T14:04:58.911 INFO:tasks.rados.rados.0.smithi017.stdout:661: finishing write tid 5 to smithi017107884-12 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:661: oid 12 updating version 0 to 109 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:661: oid 12 version 109 is already newer than 109 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:661: oid 12 version 109 is already newer than 109 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:661: oid 12 version 109 is already newer than 106 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 109 (ObjNum 323 snap 79 seq_num 323) dirty exists 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:661: left oid 12 (ObjNum 323 snap 79 seq_num 323) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:653: done (8 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:655: done (7 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:656: done (6 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:657: done (5 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:658: done (4 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:659: done (3 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:660: done (2 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:661: done (1 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:663: done (0 left) 2023-03-29T14:04:58.912 INFO:tasks.rados.rados.0.smithi017.stdout:664: delete oid 1 current snap is 79 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:664: done (0 left) 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:665: read oid 17 snap -1 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:665: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:666: read oid 6 snap -1 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:666: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:667: read oid 42 snap -1 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:667: expect (ObjNum 298 snap 78 seq_num 298) 2023-03-29T14:04:58.913 INFO:tasks.rados.rados.0.smithi017.stdout:668: delete oid 14 current snap is 79 2023-03-29T14:04:58.936 INFO:tasks.rados.rados.0.smithi017.stdout:665: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:04:59.179 INFO:tasks.rados.rados.0.smithi017.stdout:666: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:04:59.308 INFO:tasks.rados.rados.0.smithi017.stdout:667: expect (ObjNum 298 snap 78 seq_num 298) 2023-03-29T14:04:59.426 INFO:tasks.rados.rados.0.smithi017.stdout:665: done (3 left) 2023-03-29T14:04:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:666: done (2 left) 2023-03-29T14:04:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:667: done (1 left) 2023-03-29T14:04:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:668: done (0 left) 2023-03-29T14:04:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:669: read oid 2 snap 71 2023-03-29T14:04:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:669: expect (ObjNum 264 snap 70 seq_num 264) 2023-03-29T14:04:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:670: delete oid 3 current snap is 79 2023-03-29T14:04:59.440 INFO:tasks.rados.rados.0.smithi017.stdout:669: expect (ObjNum 264 snap 70 seq_num 264) 2023-03-29T14:04:59.536 INFO:tasks.rados.rados.0.smithi017.stdout:669: done (1 left) 2023-03-29T14:04:59.536 INFO:tasks.rados.rados.0.smithi017.stdout:670: done (0 left) 2023-03-29T14:04:59.537 INFO:tasks.rados.rados.0.smithi017.stdout:671: read oid 32 snap -1 2023-03-29T14:04:59.537 INFO:tasks.rados.rados.0.smithi017.stdout:671: expect (ObjNum 312 snap 79 seq_num 312) 2023-03-29T14:04:59.537 INFO:tasks.rados.rados.0.smithi017.stdout:672: delete oid 7 current snap is 79 2023-03-29T14:04:59.538 INFO:tasks.rados.rados.0.smithi017.stdout:672: done (1 left) 2023-03-29T14:04:59.538 INFO:tasks.rados.rados.0.smithi017.stdout:673: snap_remove snap 75 2023-03-29T14:05:00.116 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:00.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:00.251+0000 7fe2e6c58700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:00.258 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:00.255+0000 7fe2e6c58700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:00.262 INFO:tasks.rados.rados.0.smithi017.stdout:673: done (1 left) 2023-03-29T14:05:00.262 INFO:tasks.rados.rados.0.smithi017.stdout:674: snap_create 2023-03-29T14:05:00.262 INFO:tasks.rados.rados.0.smithi017.stdout:671: expect (ObjNum 312 snap 79 seq_num 312) 2023-03-29T14:05:01.264 INFO:tasks.rados.rados.0.smithi017.stdout:671: done (1 left) 2023-03-29T14:05:01.264 INFO:tasks.rados.rados.0.smithi017.stdout:674: done (0 left) 2023-03-29T14:05:01.264 INFO:tasks.rados.rados.0.smithi017.stdout:675: read oid 9 snap -1 2023-03-29T14:05:01.264 INFO:tasks.rados.rados.0.smithi017.stdout:675: expect (ObjNum 321 snap 79 seq_num 321) 2023-03-29T14:05:01.264 INFO:tasks.rados.rados.0.smithi017.stdout:676: delete oid 16 current snap is 80 2023-03-29T14:05:01.381 INFO:tasks.rados.rados.0.smithi017.stdout:676: done (1 left) 2023-03-29T14:05:01.382 INFO:tasks.rados.rados.0.smithi017.stdout:677: read oid 18 snap 79 2023-03-29T14:05:01.382 INFO:tasks.rados.rados.0.smithi017.stdout:677: expect deleted 2023-03-29T14:05:01.382 INFO:tasks.rados.rados.0.smithi017.stdout:678: read oid 16 snap 53 2023-03-29T14:05:01.382 INFO:tasks.rados.rados.0.smithi017.stdout:678: expect (ObjNum 175 snap 33 seq_num 175) 2023-03-29T14:05:01.382 INFO:tasks.rados.rados.0.smithi017.stdout:679: snap_remove snap 71 2023-03-29T14:05:02.271 INFO:tasks.rados.rados.0.smithi017.stdout:679: done (3 left) 2023-03-29T14:05:02.272 INFO:tasks.rados.rados.0.smithi017.stdout:680: delete oid 39 current snap is 80 2023-03-29T14:05:02.272 INFO:tasks.rados.rados.0.smithi017.stdout:675: expect (ObjNum 321 snap 79 seq_num 321) 2023-03-29T14:05:02.359 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:02.491+0000 7f29fa072700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:02.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:02.496+0000 7f29fa072700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:02.534 INFO:tasks.rados.rados.0.smithi017.stdout:678: expect (ObjNum 175 snap 33 seq_num 175) 2023-03-29T14:05:02.652 INFO:tasks.rados.rados.0.smithi017.stdout:675: done (3 left) 2023-03-29T14:05:02.652 INFO:tasks.rados.rados.0.smithi017.stdout:677: done (2 left) 2023-03-29T14:05:02.652 INFO:tasks.rados.rados.0.smithi017.stdout:678: done (1 left) 2023-03-29T14:05:02.652 INFO:tasks.rados.rados.0.smithi017.stdout:680: done (0 left) 2023-03-29T14:05:02.652 INFO:tasks.rados.rados.0.smithi017.stdout:681: snap_create 2023-03-29T14:05:03.230 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:03.230 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:03.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:03.273 INFO:tasks.rados.rados.0.smithi017.stdout:681: done (0 left) 2023-03-29T14:05:03.273 INFO:tasks.rados.rados.0.smithi017.stdout:682: write (excl) oid 1 current snap is 81 2023-03-29T14:05:03.273 INFO:tasks.rados.rados.0.smithi017.stdout:682: seq_num 331 ranges {763117=710598,2136128=720101,3390187=172930} 2023-03-29T14:05:03.280 INFO:tasks.rados.rados.0.smithi017.stdout:682: writing smithi017107884-1 from 763117 to 1473715 tid 1 2023-03-29T14:05:03.289 INFO:tasks.rados.rados.0.smithi017.stdout:682: writing smithi017107884-1 from 2136128 to 2856229 tid 2 2023-03-29T14:05:03.293 INFO:tasks.rados.rados.0.smithi017.stdout:682: writing smithi017107884-1 from 3390187 to 3563117 tid 3 2023-03-29T14:05:03.293 INFO:tasks.rados.rados.0.smithi017.stdout:683: write oid 23 current snap is 81 2023-03-29T14:05:03.293 INFO:tasks.rados.rados.0.smithi017.stdout:683: seq_num 332 ranges {779924=785847,2078257=580112,3301257=278667} 2023-03-29T14:05:03.303 INFO:tasks.rados.rados.0.smithi017.stdout:683: writing smithi017107884-23 from 779924 to 1565771 tid 1 2023-03-29T14:05:03.308 INFO:tasks.rados.rados.0.smithi017.stdout:683: writing smithi017107884-23 from 2078257 to 2658369 tid 2 2023-03-29T14:05:03.312 INFO:tasks.rados.rados.0.smithi017.stdout:683: writing smithi017107884-23 from 3301257 to 3579924 tid 3 2023-03-29T14:05:03.313 INFO:tasks.rados.rados.0.smithi017.stdout:684: write (excl) oid 18 current snap is 81 2023-03-29T14:05:03.313 INFO:tasks.rados.rados.0.smithi017.stdout:684: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2023-03-29T14:05:03.320 INFO:tasks.rados.rados.0.smithi017.stdout:684: writing smithi017107884-18 from 796731 to 1257827 tid 1 2023-03-29T14:05:03.326 INFO:tasks.rados.rados.0.smithi017.stdout:684: writing smithi017107884-18 from 1736739 to 2460509 tid 2 2023-03-29T14:05:03.333 INFO:tasks.rados.rados.0.smithi017.stdout:684: writing smithi017107884-18 from 2928680 to 3596731 tid 3 2023-03-29T14:05:03.333 INFO:tasks.rados.rados.0.smithi017.stdout:685: delete oid 41 current snap is 81 2023-03-29T14:05:03.370 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:03.366+0000 7f0139a01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:03.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:03.373+0000 7f0139a01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:03.402 INFO:tasks.rados.rados.0.smithi017.stdout:684: finishing write tid 1 to smithi017107884-18 2023-03-29T14:05:03.453 INFO:tasks.rados.rados.0.smithi017.stdout:685: done (3 left) 2023-03-29T14:05:03.453 INFO:tasks.rados.rados.0.smithi017.stdout:686: write (excl) oid 45 current snap is 81 2023-03-29T14:05:03.453 INFO:tasks.rados.rados.0.smithi017.stdout:686: seq_num 335 ranges {430345=727947,1570056=727439,2699879=469889} 2023-03-29T14:05:03.459 INFO:tasks.rados.rados.0.smithi017.stdout:686: writing smithi017107884-45 from 430345 to 1158292 tid 1 2023-03-29T14:05:03.465 INFO:tasks.rados.rados.0.smithi017.stdout:686: writing smithi017107884-45 from 1570056 to 2297495 tid 2 2023-03-29T14:05:03.473 INFO:tasks.rados.rados.0.smithi017.stdout:686: writing smithi017107884-45 from 2699879 to 3169768 tid 3 2023-03-29T14:05:03.474 INFO:tasks.rados.rados.0.smithi017.stdout:687: delete oid 26 current snap is 81 2023-03-29T14:05:03.474 INFO:tasks.rados.rados.0.smithi017.stdout:682: finishing write tid 1 to smithi017107884-1 2023-03-29T14:05:03.474 INFO:tasks.rados.rados.0.smithi017.stdout:684: finishing write tid 2 to smithi017107884-18 2023-03-29T14:05:03.485 INFO:tasks.rados.rados.0.smithi017.stdout:684: finishing write tid 3 to smithi017107884-18 2023-03-29T14:05:03.498 INFO:tasks.rados.rados.0.smithi017.stdout:684: finishing write tid 4 to smithi017107884-18 2023-03-29T14:05:03.505 INFO:tasks.rados.rados.0.smithi017.stdout:684: finishing write tid 5 to smithi017107884-18 2023-03-29T14:05:03.505 INFO:tasks.rados.rados.0.smithi017.stdout:684: oid 18 updating version 0 to 69 2023-03-29T14:05:03.505 INFO:tasks.rados.rados.0.smithi017.stdout:684: oid 18 version 69 is already newer than 69 2023-03-29T14:05:03.505 INFO:tasks.rados.rados.0.smithi017.stdout:684: oid 18 version 69 is already newer than 69 2023-03-29T14:05:03.505 INFO:tasks.rados.rados.0.smithi017.stdout:684: oid 18 version 69 is already newer than 66 2023-03-29T14:05:03.506 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 69 (ObjNum 333 snap 81 seq_num 333) dirty exists 2023-03-29T14:05:03.506 INFO:tasks.rados.rados.0.smithi017.stdout:684: left oid 18 (ObjNum 333 snap 81 seq_num 333) 2023-03-29T14:05:03.508 INFO:tasks.rados.rados.0.smithi017.stdout:683: finishing write tid 1 to smithi017107884-23 2023-03-29T14:05:03.533 INFO:tasks.rados.rados.0.smithi017.stdout:682: finishing write tid 2 to smithi017107884-1 2023-03-29T14:05:03.542 INFO:tasks.rados.rados.0.smithi017.stdout:686: finishing write tid 1 to smithi017107884-45 2023-03-29T14:05:03.542 INFO:tasks.rados.rados.0.smithi017.stdout:683: finishing write tid 2 to smithi017107884-23 2023-03-29T14:05:03.542 INFO:tasks.rados.rados.0.smithi017.stdout:684: done (4 left) 2023-03-29T14:05:03.542 INFO:tasks.rados.rados.0.smithi017.stdout:687: done (3 left) 2023-03-29T14:05:03.543 INFO:tasks.rados.rados.0.smithi017.stdout:688: snap_remove snap 70 2023-03-29T14:05:03.582 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:03.594 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2023-03-29T14:05:03.270461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":199000710,"num_objects":438,"num_object_clones":334,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":34,"num_read":1839,"num_read_kb":1001343,"num_write":2416,"num_write_kb":623998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1176,"ondisk_log_size":1176,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":862588,"kb_used_data":810780,"kb_used_omap":0,"kb_used_meta":51776,"kb_avail":376624772,"statfs":{"total":386547056640,"available":385663766528,"internally_reserved":0,"allocated":830238720,"data_stored":1001986319,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53018624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4302724,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":90,"num_read_kb":77912,"num_write":40,"num_write_kb":11984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013447"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:05:01.390653+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:05:01.390653+0000","last_peered":"2023-03-29T14:05:01.390653+0000","last_clean":"2023-03-29T14:05:01.390653+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:05:01.390653+0000","last_undegraded":"2023-03-29T14:05:01.390653+0000","last_fullsized":"2023-03-29T14:05:01.390653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":89885,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"116'12","reported_seq":162,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:57.333889+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:04:57.333889+0000","last_peered":"2023-03-29T14:04:57.333889+0000","last_clean":"2023-03-29T14:04:57.333889+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:04:57.333889+0000","last_undegraded":"2023-03-29T14:04:57.333889+0000","last_fullsized":"2023-03-29T14:04:57.333889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731220,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":17989,"num_write":26,"num_write_kb":7712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"164'95","reported_seq":346,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.933271+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:04:58.933271+0000","last_peered":"2023-03-29T14:04:58.933271+0000","last_clean":"2023-03-29T14:04:58.933271+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:04:58.933271+0000","last_undegraded":"2023-03-29T14:04:58.933271+0000","last_fullsized":"2023-03-29T14:04:58.933271+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756744,"num_objects":32,"num_object_clones":22,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":127,"num_read_kb":84887,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"170'97","reported_seq":352,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:05:01.382632+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:01.382632+0000","last_peered":"2023-03-29T14:05:01.382632+0000","last_clean":"2023-03-29T14:05:01.382632+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:01.382632+0000","last_undegraded":"2023-03-29T14:05:01.382632+0000","last_fullsized":"2023-03-29T14:05:01.382632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12807066,"num_objects":44,"num_object_clones":36,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":153,"num_read_kb":86493,"num_write":188,"num_write_kb":52580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"161'90","reported_seq":336,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:57.330376+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:04:57.330376+0000","last_peered":"2023-03-29T14:04:57.330376+0000","last_clean":"2023-03-29T14:04:57.330376+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:04:57.330376+0000","last_undegraded":"2023-03-29T14:04:57.330376+0000","last_fullsized":"2023-03-29T14:04:57.330376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20941786,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":71227,"num_write":184,"num_write_kb":54224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":130,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.278552+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:04:58.278552+0000","last_peered":"2023-03-29T14:04:58.278552+0000","last_clean":"2023-03-29T14:04:58.278552+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:04:58.278552+0000","last_undegraded":"2023-03-29T14:04:58.278552+0000","last_fullsized":"2023-03-29T14:04:58.278552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"171'64","reported_seq":274,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.945745+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:04:58.945745+0000","last_peered":"2023-03-29T14:04:58.945745+0000","last_clean":"2023-03-29T14:04:58.945745+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:04:58.945745+0000","last_undegraded":"2023-03-29T14:04:58.945745+0000","last_fullsized":"2023-03-29T14:04:58.945745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":100,"num_read_kb":51653,"num_write":116,"num_write_kb":32248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":249,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:59.538542+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:04:59.538542+0000","last_peered":"2023-03-29T14:04:59.538542+0000","last_clean":"2023-03-29T14:04:59.538542+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:04:59.538542+0000","last_undegraded":"2023-03-29T14:04:59.538542+0000","last_fullsized":"2023-03-29T14:04:59.538542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":74,"num_read_kb":41710,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:57.324820+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:04:57.324820+0000","last_peered":"2023-03-29T14:04:57.324820+0000","last_clean":"2023-03-29T14:04:57.324820+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:04:57.324820+0000","last_undegraded":"2023-03-29T14:04:57.324820+0000","last_fullsized":"2023-03-29T14:04:57.324820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":186,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.341297+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:58.341297+0000","last_peered":"2023-03-29T14:04:58.341297+0000","last_clean":"2023-03-29T14:04:58.341297+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:58.341297+0000","last_undegraded":"2023-03-29T14:04:58.341297+0000","last_fullsized":"2023-03-29T14:04:58.341297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.279077+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:04:58.279077+0000","last_peered":"2023-03-29T14:04:58.279077+0000","last_clean":"2023-03-29T14:04:58.279077+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:04:58.279077+0000","last_undegraded":"2023-03-29T14:04:58.279077+0000","last_fullsized":"2023-03-29T14:04:58.279077+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":257,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.275372+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:04:58.275372+0000","last_peered":"2023-03-29T14:04:58.275372+0000","last_clean":"2023-03-29T14:04:58.275372+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:04:58.275372+0000","last_undegraded":"2023-03-29T14:04:58.275372+0000","last_fullsized":"2023-03-29T14:04:58.275372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.343794+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:58.343794+0000","last_peered":"2023-03-29T14:04:58.343794+0000","last_clean":"2023-03-29T14:04:58.343794+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:58.343794+0000","last_undegraded":"2023-03-29T14:04:58.343794+0000","last_fullsized":"2023-03-29T14:04:58.343794+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":284,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.375079+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:04:58.375079+0000","last_peered":"2023-03-29T14:04:58.375079+0000","last_clean":"2023-03-29T14:04:58.375079+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:04:58.375079+0000","last_undegraded":"2023-03-29T14:04:58.375079+0000","last_fullsized":"2023-03-29T14:04:58.375079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.277972+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:04:58.277972+0000","last_peered":"2023-03-29T14:04:58.277972+0000","last_clean":"2023-03-29T14:04:58.277972+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:04:58.277972+0000","last_undegraded":"2023-03-29T14:04:58.277972+0000","last_fullsized":"2023-03-29T14:04:58.277972+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":358,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:57.332450+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:04:57.332450+0000","last_peered":"2023-03-29T14:04:57.332450+0000","last_clean":"2023-03-29T14:04:57.332450+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:04:57.332450+0000","last_undegraded":"2023-03-29T14:04:57.332450+0000","last_fullsized":"2023-03-29T14:04:57.332450+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"168'66","reported_seq":304,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.941193+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:04:58.941193+0000","last_peered":"2023-03-29T14:04:58.941193+0000","last_clean":"2023-03-29T14:04:58.941193+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:04:58.941193+0000","last_undegraded":"2023-03-29T14:04:58.941193+0000","last_fullsized":"2023-03-29T14:04:58.941193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17311210,"num_objects":26,"num_object_clones":18,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":144,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.276888+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:04:58.276888+0000","last_peered":"2023-03-29T14:04:58.276888+0000","last_clean":"2023-03-29T14:04:58.276888+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:04:58.276888+0000","last_undegraded":"2023-03-29T14:04:58.276888+0000","last_fullsized":"2023-03-29T14:04:58.276888+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"171'149","reported_seq":484,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-03-29T14:04:59.549688+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:04:59.549688+0000","last_peered":"2023-03-29T14:04:59.549688+0000","last_clean":"2023-03-29T14:04:59.549688+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:04:59.549688+0000","last_undegraded":"2023-03-29T14:04:59.549688+0000","last_fullsized":"2023-03-29T14:04:59.549688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23731682,"num_objects":56,"num_object_clones":42,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":251,"num_read_kb":159768,"num_write":292,"num_write_kb":84028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.365723+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:04:58.365723+0000","last_peered":"2023-03-29T14:04:58.365723+0000","last_clean":"2023-03-29T14:04:58.365723+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:04:58.365723+0000","last_undegraded":"2023-03-29T14:04:58.365723+0000","last_fullsized":"2023-03-29T14:04:58.365723+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":253,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:57.307590+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:04:57.307590+0000","last_peered":"2023-03-29T14:04:57.307590+0000","last_clean":"2023-03-29T14:04:57.307590+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:04:57.307590+0000","last_undegraded":"2023-03-29T14:04:57.307590+0000","last_fullsized":"2023-03-29T14:04:57.307590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'109","reported_seq":373,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:57.542465+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:04:57.542465+0000","last_peered":"2023-03-29T14:04:57.542465+0000","last_clean":"2023-03-29T14:04:57.542465+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:04:57.542465+0000","last_undegraded":"2023-03-29T14:04:57.542465+0000","last_fullsized":"2023-03-29T14:04:57.542465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23294766,"num_objects":42,"num_object_clones":32,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":150,"num_read_kb":83168,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":134,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.374444+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:04:58.374444+0000","last_peered":"2023-03-29T14:04:58.374444+0000","last_clean":"2023-03-29T14:04:58.374444+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:04:58.374444+0000","last_undegraded":"2023-03-29T14:04:58.374444+0000","last_fullsized":"2023-03-29T14:04:58.374444+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.282743+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:04:58.282743+0000","last_peered":"2023-03-29T14:04:58.282743+0000","last_clean":"2023-03-29T14:04:58.282743+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:04:58.282743+0000","last_undegraded":"2023-03-29T14:04:58.282743+0000","last_fullsized":"2023-03-29T14:04:58.282743+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"169'92","reported_seq":334,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.912615+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:04:58.912615+0000","last_peered":"2023-03-29T14:04:58.912615+0000","last_clean":"2023-03-29T14:04:58.912615+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:04:58.912615+0000","last_undegraded":"2023-03-29T14:04:58.912615+0000","last_fullsized":"2023-03-29T14:04:58.912615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12336470,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":125,"num_read_kb":67588,"num_write":176,"num_write_kb":50432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":197819974,"num_objects":434,"num_object_clones":334,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":34,"num_read":1697,"num_read_kb":1001223,"num_write":2142,"num_write_kb":620210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1079,"ondisk_log_size":1079,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":220480,"kb_used_data":207736,"kb_used_omap":0,"kb_used_meta":12736,"kb_avail":94151360,"statfs":{"total":96636764160,"available":96410992640,"internally_reserved":0,"allocated":212721664,"data_stored":255303190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13041664},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":253624,"kb_used_data":239984,"kb_used_omap":0,"kb_used_meta":13632,"kb_avail":94118216,"statfs":{"total":96636764160,"available":96377053184,"internally_reserved":0,"allocated":245743616,"data_stored":299083285,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13959168},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":232540,"kb_used_data":219220,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94139300,"statfs":{"total":96636764160,"available":96398643200,"internally_reserved":0,"allocated":224481280,"data_stored":274367619,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:04.280 INFO:tasks.rados.rados.0.smithi017.stdout:688: done (3 left) 2023-03-29T14:05:04.280 INFO:tasks.rados.rados.0.smithi017.stdout:689: write oid 38 current snap is 81 2023-03-29T14:05:04.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: finishing write tid 3 to smithi017107884-1 2023-03-29T14:05:04.281 INFO:tasks.rados.rados.0.smithi017.stdout:689: seq_num 337 ranges {463959=478445,1570667=731108,3038372=492433} 2023-03-29T14:05:04.285 INFO:tasks.rados.rados.0.smithi017.stdout:689: writing smithi017107884-38 from 463959 to 942404 tid 1 2023-03-29T14:05:04.293 INFO:tasks.rados.rados.0.smithi017.stdout:689: writing smithi017107884-38 from 1570667 to 2301775 tid 2 2023-03-29T14:05:04.300 INFO:tasks.rados.rados.0.smithi017.stdout:689: writing smithi017107884-38 from 3038372 to 3530805 tid 3 2023-03-29T14:05:04.300 INFO:tasks.rados.rados.0.smithi017.stdout:690: snap_remove snap 80 2023-03-29T14:05:04.567 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:04.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:04.695+0000 7f1aab48c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:04.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:04.699+0000 7f1aab48c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:690: done (4 left) 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:691: snap_create 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: finishing write tid 4 to smithi017107884-1 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: finishing write tid 5 to smithi017107884-1 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: oid 1 updating version 0 to 97 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: oid 1 version 97 is already newer than 97 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: oid 1 version 97 is already newer than 97 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: oid 1 version 97 is already newer than 94 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 97 (ObjNum 331 snap 81 seq_num 331) dirty exists 2023-03-29T14:05:05.281 INFO:tasks.rados.rados.0.smithi017.stdout:682: left oid 1 (ObjNum 331 snap 81 seq_num 331) 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: finishing write tid 3 to smithi017107884-23 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: finishing write tid 4 to smithi017107884-23 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: finishing write tid 2 to smithi017107884-45 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: finishing write tid 5 to smithi017107884-23 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: oid 23 updating version 0 to 151 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: oid 23 updating version 151 to 153 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: oid 23 version 153 is already newer than 153 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: oid 23 version 153 is already newer than 153 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 153 (ObjNum 332 snap 81 seq_num 332) dirty exists 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:683: left oid 23 (ObjNum 332 snap 81 seq_num 332) 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: finishing write tid 3 to smithi017107884-45 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: finishing write tid 4 to smithi017107884-45 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: finishing write tid 5 to smithi017107884-45 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: oid 45 updating version 0 to 95 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: oid 45 version 95 is already newer than 95 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: oid 45 version 95 is already newer than 91 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:686: oid 45 version 95 is already newer than 95 2023-03-29T14:05:05.282 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 95 (ObjNum 335 snap 81 seq_num 335) dirty exists 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:686: left oid 45 (ObjNum 335 snap 81 seq_num 335) 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: finishing write tid 1 to smithi017107884-38 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: finishing write tid 2 to smithi017107884-38 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: finishing write tid 3 to smithi017107884-38 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: finishing write tid 4 to smithi017107884-38 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: finishing write tid 5 to smithi017107884-38 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: oid 38 updating version 0 to 17 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: oid 38 version 17 is already newer than 17 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: oid 38 version 17 is already newer than 17 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: oid 38 version 17 is already newer than 15 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 17 (ObjNum 337 snap 81 seq_num 337) dirty exists 2023-03-29T14:05:05.283 INFO:tasks.rados.rados.0.smithi017.stdout:689: left oid 38 (ObjNum 337 snap 81 seq_num 337) 2023-03-29T14:05:06.289 INFO:tasks.rados.rados.0.smithi017.stdout:682: done (4 left) 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:683: done (3 left) 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:686: done (2 left) 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:689: done (1 left) 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:691: done (0 left) 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:692: read oid 19 snap -1 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:692: expect (ObjNum 317 snap 79 seq_num 317) 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:693: read oid 30 snap -1 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:693: expect deleted 2023-03-29T14:05:06.290 INFO:tasks.rados.rados.0.smithi017.stdout:694: delete oid 42 current snap is 82 2023-03-29T14:05:06.357 INFO:tasks.rados.rados.0.smithi017.stdout:692: expect (ObjNum 317 snap 79 seq_num 317) 2023-03-29T14:05:06.489 INFO:tasks.rados.rados.0.smithi017.stdout:692: done (2 left) 2023-03-29T14:05:06.489 INFO:tasks.rados.rados.0.smithi017.stdout:694: done (1 left) 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:695: read oid 13 snap 53 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:695: expect (ObjNum 215 snap 47 seq_num 215) 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:696: read oid 4 snap -1 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:696: expect deleted 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:697: read oid 19 snap -1 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:697: expect (ObjNum 317 snap 79 seq_num 317) 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:698: read oid 9 snap -1 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:698: expect (ObjNum 321 snap 79 seq_num 321) 2023-03-29T14:05:06.490 INFO:tasks.rados.rados.0.smithi017.stdout:699: snap_remove snap 81 2023-03-29T14:05:06.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:06.902 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:06.901+0000 7fc231ed5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:06.905 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:06.905+0000 7fc231ed5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:07.289 INFO:tasks.rados.rados.0.smithi017.stdout:699: done (5 left) 2023-03-29T14:05:07.289 INFO:tasks.rados.rados.0.smithi017.stdout:700: read oid 28 snap 79 2023-03-29T14:05:07.289 INFO:tasks.rados.rados.0.smithi017.stdout:700: expect (ObjNum 182 snap 34 seq_num 182) 2023-03-29T14:05:07.289 INFO:tasks.rados.rados.0.smithi017.stdout:701: delete oid 2 current snap is 82 2023-03-29T14:05:07.289 INFO:tasks.rados.rados.0.smithi017.stdout:697: expect (ObjNum 317 snap 79 seq_num 317) 2023-03-29T14:05:07.509 INFO:tasks.rados.rados.0.smithi017.stdout:698: expect (ObjNum 321 snap 79 seq_num 321) 2023-03-29T14:05:07.903 INFO:tasks.rados.rados.0.smithi017.stdout:695: expect (ObjNum 215 snap 47 seq_num 215) 2023-03-29T14:05:08.374 INFO:tasks.rados.rados.0.smithi017.stdout:700: expect (ObjNum 182 snap 34 seq_num 182) 2023-03-29T14:05:08.599 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:08.600 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:08.600 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:08.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:08.728+0000 7feed9608700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:08.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:08.735+0000 7feed9608700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:08.940 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:08.942 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:08.953 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2023-03-29T14:05:08.201916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197925128,"num_objects":450,"num_object_clones":346,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":1861,"num_read_kb":1010567,"num_write":2466,"num_write_kb":641058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1210,"ondisk_log_size":1210,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":882064,"kb_used_data":829872,"kb_used_omap":0,"kb_used_meta":52160,"kb_avail":376605296,"statfs":{"total":386547056640,"available":385643823104,"internally_reserved":0,"allocated":849788928,"data_stored":1023513942,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53411840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1075582,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":9224,"num_write":50,"num_write_kb":17060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.923840"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:05:01.390653+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:05:01.390653+0000","last_peered":"2023-03-29T14:05:01.390653+0000","last_clean":"2023-03-29T14:05:01.390653+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:05:01.390653+0000","last_undegraded":"2023-03-29T14:05:01.390653+0000","last_fullsized":"2023-03-29T14:05:01.390653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":89885,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":174,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:04.410117+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:05:04.410117+0000","last_peered":"2023-03-29T14:05:04.410117+0000","last_clean":"2023-03-29T14:05:04.410117+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:05:04.410117+0000","last_undegraded":"2023-03-29T14:05:04.410117+0000","last_fullsized":"2023-03-29T14:05:04.410117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"174'99","reported_seq":352,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.223827+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:05.223827+0000","last_peered":"2023-03-29T14:05:05.223827+0000","last_clean":"2023-03-29T14:05:05.223827+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:05.223827+0000","last_undegraded":"2023-03-29T14:05:05.223827+0000","last_fullsized":"2023-03-29T14:05:05.223827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647350,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":127,"num_read_kb":84887,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"178'99","reported_seq":361,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.504424+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:06.504424+0000","last_peered":"2023-03-29T14:05:06.504424+0000","last_clean":"2023-03-29T14:05:06.504424+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:06.504424+0000","last_undegraded":"2023-03-29T14:05:06.504424+0000","last_fullsized":"2023-03-29T14:05:06.504424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790094,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":95712,"num_write":188,"num_write_kb":52580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"175'95","reported_seq":348,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.619966+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:03.619966+0000","last_peered":"2023-03-29T14:05:03.619966+0000","last_clean":"2023-03-29T14:05:03.619966+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:03.619966+0000","last_undegraded":"2023-03-29T14:05:03.619966+0000","last_fullsized":"2023-03-29T14:05:03.619966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790556,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":71228,"num_write":194,"num_write_kb":57986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":134,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.236650+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:05:05.236650+0000","last_peered":"2023-03-29T14:05:05.236650+0000","last_clean":"2023-03-29T14:05:05.236650+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:05:05.236650+0000","last_undegraded":"2023-03-29T14:05:05.236650+0000","last_fullsized":"2023-03-29T14:05:05.236650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'69","reported_seq":287,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.231025+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:05:05.231025+0000","last_peered":"2023-03-29T14:05:05.231025+0000","last_clean":"2023-03-29T14:05:05.231025+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:05:05.231025+0000","last_undegraded":"2023-03-29T14:05:05.231025+0000","last_fullsized":"2023-03-29T14:05:05.231025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193462,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":51654,"num_write":126,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":253,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.220974+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:05.220974+0000","last_peered":"2023-03-29T14:05:05.220974+0000","last_clean":"2023-03-29T14:05:05.220974+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:05.220974+0000","last_undegraded":"2023-03-29T14:05:05.220974+0000","last_fullsized":"2023-03-29T14:05:05.220974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":74,"num_read_kb":41710,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.352250+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:05:03.352250+0000","last_peered":"2023-03-29T14:05:03.352250+0000","last_clean":"2023-03-29T14:05:03.352250+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:05:03.352250+0000","last_undegraded":"2023-03-29T14:05:03.352250+0000","last_fullsized":"2023-03-29T14:05:03.352250+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":186,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.341297+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:58.341297+0000","last_peered":"2023-03-29T14:04:58.341297+0000","last_clean":"2023-03-29T14:04:58.341297+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:58.341297+0000","last_undegraded":"2023-03-29T14:04:58.341297+0000","last_fullsized":"2023-03-29T14:04:58.341297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.232956+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:05:05.232956+0000","last_peered":"2023-03-29T14:05:05.232956+0000","last_clean":"2023-03-29T14:05:05.232956+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:05:05.232956+0000","last_undegraded":"2023-03-29T14:05:05.232956+0000","last_fullsized":"2023-03-29T14:05:05.232956+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":259,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.368149+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:05:03.368149+0000","last_peered":"2023-03-29T14:05:03.368149+0000","last_clean":"2023-03-29T14:05:03.368149+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:05:03.368149+0000","last_undegraded":"2023-03-29T14:05:03.368149+0000","last_fullsized":"2023-03-29T14:05:03.368149+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.343794+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:58.343794+0000","last_peered":"2023-03-29T14:04:58.343794+0000","last_clean":"2023-03-29T14:04:58.343794+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:58.343794+0000","last_undegraded":"2023-03-29T14:04:58.343794+0000","last_fullsized":"2023-03-29T14:04:58.343794+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":286,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.362825+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:05:03.362825+0000","last_peered":"2023-03-29T14:05:03.362825+0000","last_clean":"2023-03-29T14:05:03.362825+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:05:03.362825+0000","last_undegraded":"2023-03-29T14:05:03.362825+0000","last_fullsized":"2023-03-29T14:05:03.362825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.226931+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:05:05.226931+0000","last_peered":"2023-03-29T14:05:05.226931+0000","last_clean":"2023-03-29T14:05:05.226931+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:05:05.226931+0000","last_undegraded":"2023-03-29T14:05:05.226931+0000","last_fullsized":"2023-03-29T14:05:05.226931+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":361,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.353687+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:05:03.353687+0000","last_peered":"2023-03-29T14:05:03.353687+0000","last_clean":"2023-03-29T14:05:03.353687+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:05:03.353687+0000","last_undegraded":"2023-03-29T14:05:03.353687+0000","last_fullsized":"2023-03-29T14:05:03.353687+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":310,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.222663+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:05:05.222663+0000","last_peered":"2023-03-29T14:05:05.222663+0000","last_clean":"2023-03-29T14:05:05.222663+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:05:05.222663+0000","last_undegraded":"2023-03-29T14:05:05.222663+0000","last_fullsized":"2023-03-29T14:05:05.222663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.200005+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:05:05.200005+0000","last_peered":"2023-03-29T14:05:05.200005+0000","last_clean":"2023-03-29T14:05:05.200005+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:05:05.200005+0000","last_undegraded":"2023-03-29T14:05:05.200005+0000","last_fullsized":"2023-03-29T14:05:05.200005+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"175'153","reported_seq":495,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.575332+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:05:03.575332+0000","last_peered":"2023-03-29T14:05:03.575332+0000","last_clean":"2023-03-29T14:05:03.575332+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:05:03.575332+0000","last_undegraded":"2023-03-29T14:05:03.575332+0000","last_fullsized":"2023-03-29T14:05:03.575332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30891530,"num_objects":56,"num_object_clones":42,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":252,"num_read_kb":159769,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.319590+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:05:03.319590+0000","last_peered":"2023-03-29T14:05:03.319590+0000","last_clean":"2023-03-29T14:05:03.319590+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:05:03.319590+0000","last_undegraded":"2023-03-29T14:05:03.319590+0000","last_fullsized":"2023-03-29T14:05:03.319590+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":256,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.364704+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:03.364704+0000","last_peered":"2023-03-29T14:05:03.364704+0000","last_clean":"2023-03-29T14:05:03.364704+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:03.364704+0000","last_undegraded":"2023-03-29T14:05:03.364704+0000","last_fullsized":"2023-03-29T14:05:03.364704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":378,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.386535+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:03.386535+0000","last_peered":"2023-03-29T14:05:03.386535+0000","last_clean":"2023-03-29T14:05:03.386535+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:03.386535+0000","last_undegraded":"2023-03-29T14:05:03.386535+0000","last_fullsized":"2023-03-29T14:05:03.386535+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":152,"num_read_kb":83168,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.338964+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:05:03.338964+0000","last_peered":"2023-03-29T14:05:03.338964+0000","last_clean":"2023-03-29T14:05:03.338964+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:05:03.338964+0000","last_undegraded":"2023-03-29T14:05:03.338964+0000","last_fullsized":"2023-03-29T14:05:03.338964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-03-29T14:05:05.213687+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:05:05.213687+0000","last_peered":"2023-03-29T14:05:05.213687+0000","last_clean":"2023-03-29T14:05:05.213687+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:05:05.213687+0000","last_undegraded":"2023-03-29T14:05:05.213687+0000","last_fullsized":"2023-03-29T14:05:05.213687+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"175'97","reported_seq":345,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-03-29T14:05:03.554050+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:05:03.554050+0000","last_peered":"2023-03-29T14:05:03.554050+0000","last_clean":"2023-03-29T14:05:03.554050+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:05:03.554050+0000","last_undegraded":"2023-03-29T14:05:03.554050+0000","last_fullsized":"2023-03-29T14:05:03.554050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19462704,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":128,"num_read_kb":67589,"num_write":186,"num_write_kb":53566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":196744392,"num_objects":446,"num_object_clones":346,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":1719,"num_read_kb":1010447,"num_write":2192,"num_write_kb":637270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1113,"ondisk_log_size":1113,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":225948,"kb_used_data":213076,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94145892,"statfs":{"total":96636764160,"available":96405393408,"internally_reserved":0,"allocated":218189824,"data_stored":260898383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":260528,"kb_used_data":246760,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94111312,"statfs":{"total":96636764160,"available":96369983488,"internally_reserved":0,"allocated":252682240,"data_stored":306943518,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":239644,"kb_used_data":226196,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94132196,"statfs":{"total":96636764160,"available":96391368704,"internally_reserved":0,"allocated":231624704,"data_stored":282439816,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:10.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-03-29T14:05:13.958 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:13.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:13.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:14.087+0000 7f64ea1c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:14.093+0000 7f64ea1c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:14.296 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:14.298 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:14.310 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2023-03-29T14:05:14.131996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":192311524,"num_objects":452,"num_object_clones":348,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1891,"num_read_kb":1043231,"num_write":2466,"num_write_kb":641058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1212,"ondisk_log_size":1212,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":882100,"kb_used_data":829908,"kb_used_omap":0,"kb_used_meta":52160,"kb_avail":376605260,"statfs":{"total":386547056640,"available":385643786240,"internally_reserved":0,"allocated":849825792,"data_stored":1023547223,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53411840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20975268,"num_objects":14,"num_object_clones":14,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":48,"num_read_kb":41886,"num_write":30,"num_write_kb":10710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.845363"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:05:01.390653+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:05:01.390653+0000","last_peered":"2023-03-29T14:05:01.390653+0000","last_clean":"2023-03-29T14:05:01.390653+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:05:01.390653+0000","last_undegraded":"2023-03-29T14:05:01.390653+0000","last_fullsized":"2023-03-29T14:05:01.390653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":89885,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.422893+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:05:06.422893+0000","last_peered":"2023-03-29T14:05:06.422893+0000","last_clean":"2023-03-29T14:05:06.422893+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:05:06.422893+0000","last_undegraded":"2023-03-29T14:05:06.422893+0000","last_fullsized":"2023-03-29T14:05:06.422893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"174'99","reported_seq":354,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.319480+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:07.319480+0000","last_peered":"2023-03-29T14:05:07.319480+0000","last_clean":"2023-03-29T14:05:07.319480+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:07.319480+0000","last_undegraded":"2023-03-29T14:05:07.319480+0000","last_fullsized":"2023-03-29T14:05:07.319480+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647350,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":127,"num_read_kb":84887,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"178'99","reported_seq":363,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.298755+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:07.298755+0000","last_peered":"2023-03-29T14:05:07.298755+0000","last_clean":"2023-03-29T14:05:07.298755+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:07.298755+0000","last_undegraded":"2023-03-29T14:05:07.298755+0000","last_fullsized":"2023-03-29T14:05:07.298755+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790094,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":95712,"num_write":188,"num_write_kb":52580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"175'95","reported_seq":350,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411811+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:06.411811+0000","last_peered":"2023-03-29T14:05:06.411811+0000","last_clean":"2023-03-29T14:05:06.411811+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:06.411811+0000","last_undegraded":"2023-03-29T14:05:06.411811+0000","last_fullsized":"2023-03-29T14:05:06.411811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790556,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":71228,"num_write":194,"num_write_kb":57986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":136,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.317388+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:05:07.317388+0000","last_peered":"2023-03-29T14:05:07.317388+0000","last_clean":"2023-03-29T14:05:07.317388+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:05:07.317388+0000","last_undegraded":"2023-03-29T14:05:07.317388+0000","last_fullsized":"2023-03-29T14:05:07.317388+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'69","reported_seq":289,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.304032+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:05:07.304032+0000","last_peered":"2023-03-29T14:05:07.304032+0000","last_clean":"2023-03-29T14:05:07.304032+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:05:07.304032+0000","last_undegraded":"2023-03-29T14:05:07.304032+0000","last_fullsized":"2023-03-29T14:05:07.304032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193462,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":51654,"num_write":126,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":258,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.329601+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:07.329601+0000","last_peered":"2023-03-29T14:05:07.329601+0000","last_clean":"2023-03-29T14:05:07.329601+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:07.329601+0000","last_undegraded":"2023-03-29T14:05:07.329601+0000","last_fullsized":"2023-03-29T14:05:07.329601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.425178+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:05:06.425178+0000","last_peered":"2023-03-29T14:05:06.425178+0000","last_clean":"2023-03-29T14:05:06.425178+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:05:06.425178+0000","last_undegraded":"2023-03-29T14:05:06.425178+0000","last_fullsized":"2023-03-29T14:05:06.425178+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":186,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.341297+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:58.341297+0000","last_peered":"2023-03-29T14:04:58.341297+0000","last_clean":"2023-03-29T14:04:58.341297+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:58.341297+0000","last_undegraded":"2023-03-29T14:04:58.341297+0000","last_fullsized":"2023-03-29T14:04:58.341297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":139,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.317942+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:05:07.317942+0000","last_peered":"2023-03-29T14:05:07.317942+0000","last_clean":"2023-03-29T14:05:07.317942+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:05:07.317942+0000","last_undegraded":"2023-03-29T14:05:07.317942+0000","last_fullsized":"2023-03-29T14:05:07.317942+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":262,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.323280+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:05:07.323280+0000","last_peered":"2023-03-29T14:05:07.323280+0000","last_clean":"2023-03-29T14:05:07.323280+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:05:07.323280+0000","last_undegraded":"2023-03-29T14:05:07.323280+0000","last_fullsized":"2023-03-29T14:05:07.323280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.343794+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:58.343794+0000","last_peered":"2023-03-29T14:04:58.343794+0000","last_clean":"2023-03-29T14:04:58.343794+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:58.343794+0000","last_undegraded":"2023-03-29T14:04:58.343794+0000","last_fullsized":"2023-03-29T14:04:58.343794+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":288,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.400364+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:05:06.400364+0000","last_peered":"2023-03-29T14:05:06.400364+0000","last_clean":"2023-03-29T14:05:06.400364+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:05:06.400364+0000","last_undegraded":"2023-03-29T14:05:06.400364+0000","last_fullsized":"2023-03-29T14:05:06.400364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":139,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.306974+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:05:07.306974+0000","last_peered":"2023-03-29T14:05:07.306974+0000","last_clean":"2023-03-29T14:05:07.306974+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:05:07.306974+0000","last_undegraded":"2023-03-29T14:05:07.306974+0000","last_fullsized":"2023-03-29T14:05:07.306974+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":363,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411126+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:05:06.411126+0000","last_peered":"2023-03-29T14:05:06.411126+0000","last_clean":"2023-03-29T14:05:06.411126+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:05:06.411126+0000","last_undegraded":"2023-03-29T14:05:06.411126+0000","last_fullsized":"2023-03-29T14:05:06.411126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":312,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.308075+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:05:07.308075+0000","last_peered":"2023-03-29T14:05:07.308075+0000","last_clean":"2023-03-29T14:05:07.308075+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:05:07.308075+0000","last_undegraded":"2023-03-29T14:05:07.308075+0000","last_fullsized":"2023-03-29T14:05:07.308075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.318629+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:05:07.318629+0000","last_peered":"2023-03-29T14:05:07.318629+0000","last_clean":"2023-03-29T14:05:07.318629+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:05:07.318629+0000","last_undegraded":"2023-03-29T14:05:07.318629+0000","last_fullsized":"2023-03-29T14:05:07.318629+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"179'155","reported_seq":500,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.397258+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:05:07.397258+0000","last_peered":"2023-03-29T14:05:07.397258+0000","last_clean":"2023-03-29T14:05:07.397258+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:05:07.397258+0000","last_undegraded":"2023-03-29T14:05:07.397258+0000","last_fullsized":"2023-03-29T14:05:07.397258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277926,"num_objects":58,"num_object_clones":44,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":254,"num_read_kb":159769,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.398509+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:05:06.398509+0000","last_peered":"2023-03-29T14:05:06.398509+0000","last_clean":"2023-03-29T14:05:06.398509+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:05:06.398509+0000","last_undegraded":"2023-03-29T14:05:06.398509+0000","last_fullsized":"2023-03-29T14:05:06.398509+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":258,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.407676+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:06.407676+0000","last_peered":"2023-03-29T14:05:06.407676+0000","last_clean":"2023-03-29T14:05:06.407676+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:06.407676+0000","last_undegraded":"2023-03-29T14:05:06.407676+0000","last_fullsized":"2023-03-29T14:05:06.407676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":383,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.531458+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:06.531458+0000","last_peered":"2023-03-29T14:05:06.531458+0000","last_clean":"2023-03-29T14:05:06.531458+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:06.531458+0000","last_undegraded":"2023-03-29T14:05:06.531458+0000","last_fullsized":"2023-03-29T14:05:06.531458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":159,"num_read_kb":92204,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.413664+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:05:06.413664+0000","last_peered":"2023-03-29T14:05:06.413664+0000","last_clean":"2023-03-29T14:05:06.413664+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:05:06.413664+0000","last_undegraded":"2023-03-29T14:05:06.413664+0000","last_fullsized":"2023-03-29T14:05:06.413664+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.302824+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:05:07.302824+0000","last_peered":"2023-03-29T14:05:07.302824+0000","last_clean":"2023-03-29T14:05:07.302824+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:05:07.302824+0000","last_undegraded":"2023-03-29T14:05:07.302824+0000","last_fullsized":"2023-03-29T14:05:07.302824+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"175'97","reported_seq":354,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.365369+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:05:07.365369+0000","last_peered":"2023-03-29T14:05:07.365369+0000","last_clean":"2023-03-29T14:05:07.365369+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:05:07.365369+0000","last_undegraded":"2023-03-29T14:05:07.365369+0000","last_fullsized":"2023-03-29T14:05:07.365369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19462704,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":83787,"num_write":186,"num_write_kb":53566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":191130788,"num_objects":448,"num_object_clones":348,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1749,"num_read_kb":1043111,"num_write":2192,"num_write_kb":637270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1115,"ondisk_log_size":1115,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":225964,"kb_used_data":213092,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94145876,"statfs":{"total":96636764160,"available":96405377024,"internally_reserved":0,"allocated":218206208,"data_stored":260913179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705848,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":260540,"kb_used_data":246772,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94111300,"statfs":{"total":96636764160,"available":96369971200,"internally_reserved":0,"allocated":252694528,"data_stored":306954621,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705848,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":239652,"kb_used_data":226204,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94132188,"statfs":{"total":96636764160,"available":96391360512,"internally_reserved":0,"allocated":231632896,"data_stored":282447198,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:15.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~6s 2023-03-29T14:05:19.318 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:19.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:05:19.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:19.447+0000 7f7defd5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:19.455 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:19.455+0000 7f7defd5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:20.306 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:05:20.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:05:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:20.445+0000 7fbf6a3cf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:20.454+0000 7fbf6a3cf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:21.073 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~11s 2023-03-29T14:05:21.310 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:05:21.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:05:21.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:21.451+0000 7f1d60668700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:21.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:21.460+0000 7f1d60668700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:05:22.326 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:05:22.462 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:22.457+0000 7f1fcb072700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:22.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:22.465+0000 7f1fcb072700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:23.209 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:05:23.222 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:05:23.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:23.350+0000 7f50a873c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:23.357+0000 7f50a873c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:23.559 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:23.573 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2023-03-29T14:05:23.205415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":192311524,"num_objects":452,"num_object_clones":348,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1891,"num_read_kb":1043231,"num_write":2466,"num_write_kb":641058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1212,"ondisk_log_size":1212,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":882116,"kb_used_data":829924,"kb_used_omap":0,"kb_used_meta":52160,"kb_avail":376605244,"statfs":{"total":386547056640,"available":385643769856,"internally_reserved":0,"allocated":849842176,"data_stored":1023562023,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53411840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.072835"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:05:01.390653+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:05:01.390653+0000","last_peered":"2023-03-29T14:05:01.390653+0000","last_clean":"2023-03-29T14:05:01.390653+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:05:01.390653+0000","last_undegraded":"2023-03-29T14:05:01.390653+0000","last_fullsized":"2023-03-29T14:05:01.390653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":89885,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.422893+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:05:06.422893+0000","last_peered":"2023-03-29T14:05:06.422893+0000","last_clean":"2023-03-29T14:05:06.422893+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:05:06.422893+0000","last_undegraded":"2023-03-29T14:05:06.422893+0000","last_fullsized":"2023-03-29T14:05:06.422893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"174'99","reported_seq":356,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.316577+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:20.316577+0000","last_peered":"2023-03-29T14:05:20.316577+0000","last_clean":"2023-03-29T14:05:20.316577+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:20.316577+0000","last_undegraded":"2023-03-29T14:05:20.316577+0000","last_fullsized":"2023-03-29T14:05:20.316577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647350,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":127,"num_read_kb":84887,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"178'99","reported_seq":365,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.329629+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:20.329629+0000","last_peered":"2023-03-29T14:05:20.329629+0000","last_clean":"2023-03-29T14:05:20.329629+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:20.329629+0000","last_undegraded":"2023-03-29T14:05:20.329629+0000","last_fullsized":"2023-03-29T14:05:20.329629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790094,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":95712,"num_write":188,"num_write_kb":52580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"175'95","reported_seq":350,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411811+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:06.411811+0000","last_peered":"2023-03-29T14:05:06.411811+0000","last_clean":"2023-03-29T14:05:06.411811+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:06.411811+0000","last_undegraded":"2023-03-29T14:05:06.411811+0000","last_fullsized":"2023-03-29T14:05:06.411811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790556,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":71228,"num_write":194,"num_write_kb":57986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":138,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.318802+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:05:20.318802+0000","last_peered":"2023-03-29T14:05:20.318802+0000","last_clean":"2023-03-29T14:05:20.318802+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:05:20.318802+0000","last_undegraded":"2023-03-29T14:05:20.318802+0000","last_fullsized":"2023-03-29T14:05:20.318802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'69","reported_seq":291,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.326627+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:05:20.326627+0000","last_peered":"2023-03-29T14:05:20.326627+0000","last_clean":"2023-03-29T14:05:20.326627+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:05:20.326627+0000","last_undegraded":"2023-03-29T14:05:20.326627+0000","last_fullsized":"2023-03-29T14:05:20.326627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193462,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":51654,"num_write":126,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":260,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.313832+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:20.313832+0000","last_peered":"2023-03-29T14:05:20.313832+0000","last_clean":"2023-03-29T14:05:20.313832+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:20.313832+0000","last_undegraded":"2023-03-29T14:05:20.313832+0000","last_fullsized":"2023-03-29T14:05:20.313832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.425178+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:05:06.425178+0000","last_peered":"2023-03-29T14:05:06.425178+0000","last_clean":"2023-03-29T14:05:06.425178+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:05:06.425178+0000","last_undegraded":"2023-03-29T14:05:06.425178+0000","last_fullsized":"2023-03-29T14:05:06.425178+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":186,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.341297+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:58.341297+0000","last_peered":"2023-03-29T14:04:58.341297+0000","last_clean":"2023-03-29T14:04:58.341297+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:58.341297+0000","last_undegraded":"2023-03-29T14:04:58.341297+0000","last_fullsized":"2023-03-29T14:04:58.341297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.324812+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:05:20.324812+0000","last_peered":"2023-03-29T14:05:20.324812+0000","last_clean":"2023-03-29T14:05:20.324812+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:05:20.324812+0000","last_undegraded":"2023-03-29T14:05:20.324812+0000","last_fullsized":"2023-03-29T14:05:20.324812+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":262,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.323280+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:05:07.323280+0000","last_peered":"2023-03-29T14:05:07.323280+0000","last_clean":"2023-03-29T14:05:07.323280+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:05:07.323280+0000","last_undegraded":"2023-03-29T14:05:07.323280+0000","last_fullsized":"2023-03-29T14:05:07.323280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.343794+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:58.343794+0000","last_peered":"2023-03-29T14:04:58.343794+0000","last_clean":"2023-03-29T14:04:58.343794+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:58.343794+0000","last_undegraded":"2023-03-29T14:04:58.343794+0000","last_fullsized":"2023-03-29T14:04:58.343794+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":288,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.400364+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:05:06.400364+0000","last_peered":"2023-03-29T14:05:06.400364+0000","last_clean":"2023-03-29T14:05:06.400364+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:05:06.400364+0000","last_undegraded":"2023-03-29T14:05:06.400364+0000","last_fullsized":"2023-03-29T14:05:06.400364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.315411+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:05:20.315411+0000","last_peered":"2023-03-29T14:05:20.315411+0000","last_clean":"2023-03-29T14:05:20.315411+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:05:20.315411+0000","last_undegraded":"2023-03-29T14:05:20.315411+0000","last_fullsized":"2023-03-29T14:05:20.315411+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":363,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411126+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:05:06.411126+0000","last_peered":"2023-03-29T14:05:06.411126+0000","last_clean":"2023-03-29T14:05:06.411126+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:05:06.411126+0000","last_undegraded":"2023-03-29T14:05:06.411126+0000","last_fullsized":"2023-03-29T14:05:06.411126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":314,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.308843+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:05:20.308843+0000","last_peered":"2023-03-29T14:05:20.308843+0000","last_clean":"2023-03-29T14:05:20.308843+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:05:20.308843+0000","last_undegraded":"2023-03-29T14:05:20.308843+0000","last_fullsized":"2023-03-29T14:05:20.308843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.318254+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:05:20.318254+0000","last_peered":"2023-03-29T14:05:20.318254+0000","last_clean":"2023-03-29T14:05:20.318254+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:05:20.318254+0000","last_undegraded":"2023-03-29T14:05:20.318254+0000","last_fullsized":"2023-03-29T14:05:20.318254+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"179'155","reported_seq":500,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.397258+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:05:07.397258+0000","last_peered":"2023-03-29T14:05:07.397258+0000","last_clean":"2023-03-29T14:05:07.397258+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:05:07.397258+0000","last_undegraded":"2023-03-29T14:05:07.397258+0000","last_fullsized":"2023-03-29T14:05:07.397258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277926,"num_objects":58,"num_object_clones":44,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":254,"num_read_kb":159769,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.398509+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:05:06.398509+0000","last_peered":"2023-03-29T14:05:06.398509+0000","last_clean":"2023-03-29T14:05:06.398509+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:05:06.398509+0000","last_undegraded":"2023-03-29T14:05:06.398509+0000","last_fullsized":"2023-03-29T14:05:06.398509+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":258,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.407676+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:06.407676+0000","last_peered":"2023-03-29T14:05:06.407676+0000","last_clean":"2023-03-29T14:05:06.407676+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:06.407676+0000","last_undegraded":"2023-03-29T14:05:06.407676+0000","last_fullsized":"2023-03-29T14:05:06.407676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":383,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.531458+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:06.531458+0000","last_peered":"2023-03-29T14:05:06.531458+0000","last_clean":"2023-03-29T14:05:06.531458+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:06.531458+0000","last_undegraded":"2023-03-29T14:05:06.531458+0000","last_fullsized":"2023-03-29T14:05:06.531458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":159,"num_read_kb":92204,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.413664+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:05:06.413664+0000","last_peered":"2023-03-29T14:05:06.413664+0000","last_clean":"2023-03-29T14:05:06.413664+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:05:06.413664+0000","last_undegraded":"2023-03-29T14:05:06.413664+0000","last_fullsized":"2023-03-29T14:05:06.413664+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.317725+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:05:20.317725+0000","last_peered":"2023-03-29T14:05:20.317725+0000","last_clean":"2023-03-29T14:05:20.317725+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:05:20.317725+0000","last_undegraded":"2023-03-29T14:05:20.317725+0000","last_fullsized":"2023-03-29T14:05:20.317725+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"175'97","reported_seq":354,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.365369+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:05:07.365369+0000","last_peered":"2023-03-29T14:05:07.365369+0000","last_clean":"2023-03-29T14:05:07.365369+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:05:07.365369+0000","last_undegraded":"2023-03-29T14:05:07.365369+0000","last_fullsized":"2023-03-29T14:05:07.365369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19462704,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":83787,"num_write":186,"num_write_kb":53566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":191130788,"num_objects":448,"num_object_clones":348,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1749,"num_read_kb":1043111,"num_write":2192,"num_write_kb":637270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1115,"ondisk_log_size":1115,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":225968,"kb_used_data":213096,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94145872,"statfs":{"total":96636764160,"available":96405372928,"internally_reserved":0,"allocated":218210304,"data_stored":260916872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":260544,"kb_used_data":246776,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94111296,"statfs":{"total":96636764160,"available":96369967104,"internally_reserved":0,"allocated":252698624,"data_stored":306958314,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":239660,"kb_used_data":226212,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94132180,"statfs":{"total":96636764160,"available":96391352320,"internally_reserved":0,"allocated":231641088,"data_stored":282454612,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:23.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:23.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:23.704+0000 7fdd3a1b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:23.710 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:23.711+0000 7fdd3a1b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:23.916 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:23.917 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:23.929 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2023-03-29T14:05:23.205415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":192311524,"num_objects":452,"num_object_clones":348,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1891,"num_read_kb":1043231,"num_write":2466,"num_write_kb":641058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1212,"ondisk_log_size":1212,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":882116,"kb_used_data":829924,"kb_used_omap":0,"kb_used_meta":52160,"kb_avail":376605244,"statfs":{"total":386547056640,"available":385643769856,"internally_reserved":0,"allocated":849842176,"data_stored":1023562023,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53411840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.072835"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:05:01.390653+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:05:01.390653+0000","last_peered":"2023-03-29T14:05:01.390653+0000","last_clean":"2023-03-29T14:05:01.390653+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:05:01.390653+0000","last_undegraded":"2023-03-29T14:05:01.390653+0000","last_fullsized":"2023-03-29T14:05:01.390653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":89885,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.422893+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:05:06.422893+0000","last_peered":"2023-03-29T14:05:06.422893+0000","last_clean":"2023-03-29T14:05:06.422893+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:05:06.422893+0000","last_undegraded":"2023-03-29T14:05:06.422893+0000","last_fullsized":"2023-03-29T14:05:06.422893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"174'99","reported_seq":356,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.316577+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:20.316577+0000","last_peered":"2023-03-29T14:05:20.316577+0000","last_clean":"2023-03-29T14:05:20.316577+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:20.316577+0000","last_undegraded":"2023-03-29T14:05:20.316577+0000","last_fullsized":"2023-03-29T14:05:20.316577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647350,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":127,"num_read_kb":84887,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"178'99","reported_seq":365,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.329629+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:20.329629+0000","last_peered":"2023-03-29T14:05:20.329629+0000","last_clean":"2023-03-29T14:05:20.329629+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:20.329629+0000","last_undegraded":"2023-03-29T14:05:20.329629+0000","last_fullsized":"2023-03-29T14:05:20.329629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790094,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":95712,"num_write":188,"num_write_kb":52580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"175'95","reported_seq":350,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411811+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:06.411811+0000","last_peered":"2023-03-29T14:05:06.411811+0000","last_clean":"2023-03-29T14:05:06.411811+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:06.411811+0000","last_undegraded":"2023-03-29T14:05:06.411811+0000","last_fullsized":"2023-03-29T14:05:06.411811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790556,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":71228,"num_write":194,"num_write_kb":57986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":138,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.318802+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:05:20.318802+0000","last_peered":"2023-03-29T14:05:20.318802+0000","last_clean":"2023-03-29T14:05:20.318802+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:05:20.318802+0000","last_undegraded":"2023-03-29T14:05:20.318802+0000","last_fullsized":"2023-03-29T14:05:20.318802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'69","reported_seq":291,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.326627+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:05:20.326627+0000","last_peered":"2023-03-29T14:05:20.326627+0000","last_clean":"2023-03-29T14:05:20.326627+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:05:20.326627+0000","last_undegraded":"2023-03-29T14:05:20.326627+0000","last_fullsized":"2023-03-29T14:05:20.326627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193462,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":51654,"num_write":126,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":260,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.313832+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:20.313832+0000","last_peered":"2023-03-29T14:05:20.313832+0000","last_clean":"2023-03-29T14:05:20.313832+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:20.313832+0000","last_undegraded":"2023-03-29T14:05:20.313832+0000","last_fullsized":"2023-03-29T14:05:20.313832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.425178+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:05:06.425178+0000","last_peered":"2023-03-29T14:05:06.425178+0000","last_clean":"2023-03-29T14:05:06.425178+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:05:06.425178+0000","last_undegraded":"2023-03-29T14:05:06.425178+0000","last_fullsized":"2023-03-29T14:05:06.425178+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":186,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.341297+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:58.341297+0000","last_peered":"2023-03-29T14:04:58.341297+0000","last_clean":"2023-03-29T14:04:58.341297+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:58.341297+0000","last_undegraded":"2023-03-29T14:04:58.341297+0000","last_fullsized":"2023-03-29T14:04:58.341297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.324812+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:05:20.324812+0000","last_peered":"2023-03-29T14:05:20.324812+0000","last_clean":"2023-03-29T14:05:20.324812+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:05:20.324812+0000","last_undegraded":"2023-03-29T14:05:20.324812+0000","last_fullsized":"2023-03-29T14:05:20.324812+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":262,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.323280+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:05:07.323280+0000","last_peered":"2023-03-29T14:05:07.323280+0000","last_clean":"2023-03-29T14:05:07.323280+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:05:07.323280+0000","last_undegraded":"2023-03-29T14:05:07.323280+0000","last_fullsized":"2023-03-29T14:05:07.323280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.343794+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:58.343794+0000","last_peered":"2023-03-29T14:04:58.343794+0000","last_clean":"2023-03-29T14:04:58.343794+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:58.343794+0000","last_undegraded":"2023-03-29T14:04:58.343794+0000","last_fullsized":"2023-03-29T14:04:58.343794+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":288,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.400364+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:05:06.400364+0000","last_peered":"2023-03-29T14:05:06.400364+0000","last_clean":"2023-03-29T14:05:06.400364+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:05:06.400364+0000","last_undegraded":"2023-03-29T14:05:06.400364+0000","last_fullsized":"2023-03-29T14:05:06.400364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.315411+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:05:20.315411+0000","last_peered":"2023-03-29T14:05:20.315411+0000","last_clean":"2023-03-29T14:05:20.315411+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:05:20.315411+0000","last_undegraded":"2023-03-29T14:05:20.315411+0000","last_fullsized":"2023-03-29T14:05:20.315411+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":363,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411126+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:05:06.411126+0000","last_peered":"2023-03-29T14:05:06.411126+0000","last_clean":"2023-03-29T14:05:06.411126+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:05:06.411126+0000","last_undegraded":"2023-03-29T14:05:06.411126+0000","last_fullsized":"2023-03-29T14:05:06.411126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":314,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.308843+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:05:20.308843+0000","last_peered":"2023-03-29T14:05:20.308843+0000","last_clean":"2023-03-29T14:05:20.308843+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:05:20.308843+0000","last_undegraded":"2023-03-29T14:05:20.308843+0000","last_fullsized":"2023-03-29T14:05:20.308843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.318254+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:05:20.318254+0000","last_peered":"2023-03-29T14:05:20.318254+0000","last_clean":"2023-03-29T14:05:20.318254+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:05:20.318254+0000","last_undegraded":"2023-03-29T14:05:20.318254+0000","last_fullsized":"2023-03-29T14:05:20.318254+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"179'155","reported_seq":500,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.397258+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:05:07.397258+0000","last_peered":"2023-03-29T14:05:07.397258+0000","last_clean":"2023-03-29T14:05:07.397258+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:05:07.397258+0000","last_undegraded":"2023-03-29T14:05:07.397258+0000","last_fullsized":"2023-03-29T14:05:07.397258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277926,"num_objects":58,"num_object_clones":44,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":254,"num_read_kb":159769,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.398509+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:05:06.398509+0000","last_peered":"2023-03-29T14:05:06.398509+0000","last_clean":"2023-03-29T14:05:06.398509+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:05:06.398509+0000","last_undegraded":"2023-03-29T14:05:06.398509+0000","last_fullsized":"2023-03-29T14:05:06.398509+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":258,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.407676+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:06.407676+0000","last_peered":"2023-03-29T14:05:06.407676+0000","last_clean":"2023-03-29T14:05:06.407676+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:06.407676+0000","last_undegraded":"2023-03-29T14:05:06.407676+0000","last_fullsized":"2023-03-29T14:05:06.407676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":383,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.531458+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:06.531458+0000","last_peered":"2023-03-29T14:05:06.531458+0000","last_clean":"2023-03-29T14:05:06.531458+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:06.531458+0000","last_undegraded":"2023-03-29T14:05:06.531458+0000","last_fullsized":"2023-03-29T14:05:06.531458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":159,"num_read_kb":92204,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.413664+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:05:06.413664+0000","last_peered":"2023-03-29T14:05:06.413664+0000","last_clean":"2023-03-29T14:05:06.413664+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:05:06.413664+0000","last_undegraded":"2023-03-29T14:05:06.413664+0000","last_fullsized":"2023-03-29T14:05:06.413664+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.317725+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:05:20.317725+0000","last_peered":"2023-03-29T14:05:20.317725+0000","last_clean":"2023-03-29T14:05:20.317725+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:05:20.317725+0000","last_undegraded":"2023-03-29T14:05:20.317725+0000","last_fullsized":"2023-03-29T14:05:20.317725+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"175'97","reported_seq":354,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.365369+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:05:07.365369+0000","last_peered":"2023-03-29T14:05:07.365369+0000","last_clean":"2023-03-29T14:05:07.365369+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:05:07.365369+0000","last_undegraded":"2023-03-29T14:05:07.365369+0000","last_fullsized":"2023-03-29T14:05:07.365369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19462704,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":83787,"num_write":186,"num_write_kb":53566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":191130788,"num_objects":448,"num_object_clones":348,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1749,"num_read_kb":1043111,"num_write":2192,"num_write_kb":637270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1115,"ondisk_log_size":1115,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":225968,"kb_used_data":213096,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94145872,"statfs":{"total":96636764160,"available":96405372928,"internally_reserved":0,"allocated":218210304,"data_stored":260916872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":260544,"kb_used_data":246776,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94111296,"statfs":{"total":96636764160,"available":96369967104,"internally_reserved":0,"allocated":252698624,"data_stored":306958314,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":239660,"kb_used_data":226212,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94132180,"statfs":{"total":96636764160,"available":96391352320,"internally_reserved":0,"allocated":231641088,"data_stored":282454612,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:23.932 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:05:23.932 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:23.933 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:24.063 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:24.060+0000 7ff70396d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:24.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:24.066+0000 7ff70396d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:24.268 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:24.281 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2023-03-29T14:05:24.133476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":192311524,"num_objects":452,"num_object_clones":348,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1891,"num_read_kb":1043231,"num_write":2466,"num_write_kb":641058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1212,"ondisk_log_size":1212,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":882116,"kb_used_data":829924,"kb_used_omap":0,"kb_used_meta":52160,"kb_avail":376605244,"statfs":{"total":386547056640,"available":385643769856,"internally_reserved":0,"allocated":849842176,"data_stored":1023562023,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53411840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000585"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:05:01.390653+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:05:01.390653+0000","last_peered":"2023-03-29T14:05:01.390653+0000","last_clean":"2023-03-29T14:05:01.390653+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:05:01.390653+0000","last_undegraded":"2023-03-29T14:05:01.390653+0000","last_fullsized":"2023-03-29T14:05:01.390653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":89885,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.422893+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:05:06.422893+0000","last_peered":"2023-03-29T14:05:06.422893+0000","last_clean":"2023-03-29T14:05:06.422893+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:05:06.422893+0000","last_undegraded":"2023-03-29T14:05:06.422893+0000","last_fullsized":"2023-03-29T14:05:06.422893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"174'99","reported_seq":356,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.316577+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:20.316577+0000","last_peered":"2023-03-29T14:05:20.316577+0000","last_clean":"2023-03-29T14:05:20.316577+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:20.316577+0000","last_undegraded":"2023-03-29T14:05:20.316577+0000","last_fullsized":"2023-03-29T14:05:20.316577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647350,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":127,"num_read_kb":84887,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"178'99","reported_seq":365,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.329629+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:20.329629+0000","last_peered":"2023-03-29T14:05:20.329629+0000","last_clean":"2023-03-29T14:05:20.329629+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:20.329629+0000","last_undegraded":"2023-03-29T14:05:20.329629+0000","last_fullsized":"2023-03-29T14:05:20.329629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790094,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":95712,"num_write":188,"num_write_kb":52580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"175'95","reported_seq":350,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411811+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:06.411811+0000","last_peered":"2023-03-29T14:05:06.411811+0000","last_clean":"2023-03-29T14:05:06.411811+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:06.411811+0000","last_undegraded":"2023-03-29T14:05:06.411811+0000","last_fullsized":"2023-03-29T14:05:06.411811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790556,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":71228,"num_write":194,"num_write_kb":57986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":138,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.318802+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:05:20.318802+0000","last_peered":"2023-03-29T14:05:20.318802+0000","last_clean":"2023-03-29T14:05:20.318802+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:05:20.318802+0000","last_undegraded":"2023-03-29T14:05:20.318802+0000","last_fullsized":"2023-03-29T14:05:20.318802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'69","reported_seq":291,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.326627+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:05:20.326627+0000","last_peered":"2023-03-29T14:05:20.326627+0000","last_clean":"2023-03-29T14:05:20.326627+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:05:20.326627+0000","last_undegraded":"2023-03-29T14:05:20.326627+0000","last_fullsized":"2023-03-29T14:05:20.326627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193462,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":51654,"num_write":126,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":260,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.313832+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:20.313832+0000","last_peered":"2023-03-29T14:05:20.313832+0000","last_clean":"2023-03-29T14:05:20.313832+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:20.313832+0000","last_undegraded":"2023-03-29T14:05:20.313832+0000","last_fullsized":"2023-03-29T14:05:20.313832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.425178+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:05:06.425178+0000","last_peered":"2023-03-29T14:05:06.425178+0000","last_clean":"2023-03-29T14:05:06.425178+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:05:06.425178+0000","last_undegraded":"2023-03-29T14:05:06.425178+0000","last_fullsized":"2023-03-29T14:05:06.425178+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":186,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.341297+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:58.341297+0000","last_peered":"2023-03-29T14:04:58.341297+0000","last_clean":"2023-03-29T14:04:58.341297+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:04:58.341297+0000","last_undegraded":"2023-03-29T14:04:58.341297+0000","last_fullsized":"2023-03-29T14:04:58.341297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.324812+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:05:20.324812+0000","last_peered":"2023-03-29T14:05:20.324812+0000","last_clean":"2023-03-29T14:05:20.324812+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:05:20.324812+0000","last_undegraded":"2023-03-29T14:05:20.324812+0000","last_fullsized":"2023-03-29T14:05:20.324812+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":262,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.323280+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:05:07.323280+0000","last_peered":"2023-03-29T14:05:07.323280+0000","last_clean":"2023-03-29T14:05:07.323280+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:05:07.323280+0000","last_undegraded":"2023-03-29T14:05:07.323280+0000","last_fullsized":"2023-03-29T14:05:07.323280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-03-29T14:04:58.343794+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:58.343794+0000","last_peered":"2023-03-29T14:04:58.343794+0000","last_clean":"2023-03-29T14:04:58.343794+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:04:58.343794+0000","last_undegraded":"2023-03-29T14:04:58.343794+0000","last_fullsized":"2023-03-29T14:04:58.343794+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":288,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.400364+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:05:06.400364+0000","last_peered":"2023-03-29T14:05:06.400364+0000","last_clean":"2023-03-29T14:05:06.400364+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:05:06.400364+0000","last_undegraded":"2023-03-29T14:05:06.400364+0000","last_fullsized":"2023-03-29T14:05:06.400364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.315411+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:05:20.315411+0000","last_peered":"2023-03-29T14:05:20.315411+0000","last_clean":"2023-03-29T14:05:20.315411+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:05:20.315411+0000","last_undegraded":"2023-03-29T14:05:20.315411+0000","last_fullsized":"2023-03-29T14:05:20.315411+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":363,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.411126+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:05:06.411126+0000","last_peered":"2023-03-29T14:05:06.411126+0000","last_clean":"2023-03-29T14:05:06.411126+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:05:06.411126+0000","last_undegraded":"2023-03-29T14:05:06.411126+0000","last_fullsized":"2023-03-29T14:05:06.411126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":314,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.308843+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:05:20.308843+0000","last_peered":"2023-03-29T14:05:20.308843+0000","last_clean":"2023-03-29T14:05:20.308843+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:05:20.308843+0000","last_undegraded":"2023-03-29T14:05:20.308843+0000","last_fullsized":"2023-03-29T14:05:20.308843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.318254+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:05:20.318254+0000","last_peered":"2023-03-29T14:05:20.318254+0000","last_clean":"2023-03-29T14:05:20.318254+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:05:20.318254+0000","last_undegraded":"2023-03-29T14:05:20.318254+0000","last_fullsized":"2023-03-29T14:05:20.318254+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"179'155","reported_seq":500,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.397258+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:05:07.397258+0000","last_peered":"2023-03-29T14:05:07.397258+0000","last_clean":"2023-03-29T14:05:07.397258+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:05:07.397258+0000","last_undegraded":"2023-03-29T14:05:07.397258+0000","last_fullsized":"2023-03-29T14:05:07.397258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277926,"num_objects":58,"num_object_clones":44,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":254,"num_read_kb":159769,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.398509+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:05:06.398509+0000","last_peered":"2023-03-29T14:05:06.398509+0000","last_clean":"2023-03-29T14:05:06.398509+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:05:06.398509+0000","last_undegraded":"2023-03-29T14:05:06.398509+0000","last_fullsized":"2023-03-29T14:05:06.398509+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":258,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.407676+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:06.407676+0000","last_peered":"2023-03-29T14:05:06.407676+0000","last_clean":"2023-03-29T14:05:06.407676+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:06.407676+0000","last_undegraded":"2023-03-29T14:05:06.407676+0000","last_fullsized":"2023-03-29T14:05:06.407676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":383,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.531458+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:06.531458+0000","last_peered":"2023-03-29T14:05:06.531458+0000","last_clean":"2023-03-29T14:05:06.531458+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:06.531458+0000","last_undegraded":"2023-03-29T14:05:06.531458+0000","last_fullsized":"2023-03-29T14:05:06.531458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":159,"num_read_kb":92204,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:06.413664+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:05:06.413664+0000","last_peered":"2023-03-29T14:05:06.413664+0000","last_clean":"2023-03-29T14:05:06.413664+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:05:06.413664+0000","last_undegraded":"2023-03-29T14:05:06.413664+0000","last_fullsized":"2023-03-29T14:05:06.413664+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-03-29T14:05:20.317725+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:05:20.317725+0000","last_peered":"2023-03-29T14:05:20.317725+0000","last_clean":"2023-03-29T14:05:20.317725+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:05:20.317725+0000","last_undegraded":"2023-03-29T14:05:20.317725+0000","last_fullsized":"2023-03-29T14:05:20.317725+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"175'97","reported_seq":354,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-03-29T14:05:07.365369+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:05:07.365369+0000","last_peered":"2023-03-29T14:05:07.365369+0000","last_clean":"2023-03-29T14:05:07.365369+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:05:07.365369+0000","last_undegraded":"2023-03-29T14:05:07.365369+0000","last_fullsized":"2023-03-29T14:05:07.365369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19462704,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":83787,"num_write":186,"num_write_kb":53566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":191130788,"num_objects":448,"num_object_clones":348,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1749,"num_read_kb":1043111,"num_write":2192,"num_write_kb":637270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1115,"ondisk_log_size":1115,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":225968,"kb_used_data":213096,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94145872,"statfs":{"total":96636764160,"available":96405372928,"internally_reserved":0,"allocated":218210304,"data_stored":260916872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":260544,"kb_used_data":246776,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94111296,"statfs":{"total":96636764160,"available":96369967104,"internally_reserved":0,"allocated":252698624,"data_stored":306958314,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":239660,"kb_used_data":226212,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94132180,"statfs":{"total":96636764160,"available":96391352320,"internally_reserved":0,"allocated":231641088,"data_stored":282454612,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:26.577 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~17s 2023-03-29T14:05:29.286 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:29.286 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:29.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:29.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:29.418+0000 7f3b66f51700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:29.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:29.426+0000 7f3b66f51700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:29.634 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:29.646 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-03-29T14:05:29.318033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":192311524,"num_objects":452,"num_object_clones":348,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1891,"num_read_kb":1043231,"num_write":2466,"num_write_kb":641058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1212,"ondisk_log_size":1212,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":882152,"kb_used_data":829960,"kb_used_omap":0,"kb_used_meta":52160,"kb_avail":376605208,"statfs":{"total":386547056640,"available":385643732992,"internally_reserved":0,"allocated":849879040,"data_stored":1023595260,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53411840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.184876"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":232,"reported_epoch":174,"state":"stale+active+clean","last_fresh":"2023-03-29T14:05:01.390653+0000","last_change":"2023-03-29T14:02:17.439635+0000","last_active":"2023-03-29T14:05:01.390653+0000","last_peered":"2023-03-29T14:05:01.390653+0000","last_clean":"2023-03-29T14:05:01.390653+0000","last_became_active":"2023-03-29T14:02:17.430273+0000","last_became_peered":"2023-03-29T14:02:17.430273+0000","last_unstale":"2023-03-29T14:05:29.318047+0000","last_undegraded":"2023-03-29T14:05:01.390653+0000","last_fullsized":"2023-03-29T14:05:01.390653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5277464,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":89885,"num_write":76,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":178,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.497660+0000","last_change":"2023-03-29T14:02:17.517876+0000","last_active":"2023-03-29T14:05:21.497660+0000","last_peered":"2023-03-29T14:05:21.497660+0000","last_clean":"2023-03-29T14:05:21.497660+0000","last_became_active":"2023-03-29T14:02:17.511265+0000","last_became_peered":"2023-03-29T14:02:17.511265+0000","last_unstale":"2023-03-29T14:05:21.497660+0000","last_undegraded":"2023-03-29T14:05:21.497660+0000","last_fullsized":"2023-03-29T14:05:21.497660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"174'99","reported_seq":358,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.321824+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:22.321824+0000","last_peered":"2023-03-29T14:05:22.321824+0000","last_clean":"2023-03-29T14:05:22.321824+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:22.321824+0000","last_undegraded":"2023-03-29T14:05:22.321824+0000","last_fullsized":"2023-03-29T14:05:22.321824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647350,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":127,"num_read_kb":84887,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"178'99","reported_seq":367,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.317409+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:22.317409+0000","last_peered":"2023-03-29T14:05:22.317409+0000","last_clean":"2023-03-29T14:05:22.317409+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:22.317409+0000","last_undegraded":"2023-03-29T14:05:22.317409+0000","last_fullsized":"2023-03-29T14:05:22.317409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790094,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":95712,"num_write":188,"num_write_kb":52580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"175'95","reported_seq":352,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.546646+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:21.546646+0000","last_peered":"2023-03-29T14:05:21.546646+0000","last_clean":"2023-03-29T14:05:21.546646+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:21.546646+0000","last_undegraded":"2023-03-29T14:05:21.546646+0000","last_fullsized":"2023-03-29T14:05:21.546646+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790556,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":71228,"num_write":194,"num_write_kb":57986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":140,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.331203+0000","last_change":"2023-03-29T14:02:17.500159+0000","last_active":"2023-03-29T14:05:22.331203+0000","last_peered":"2023-03-29T14:05:22.331203+0000","last_clean":"2023-03-29T14:05:22.331203+0000","last_became_active":"2023-03-29T14:02:17.490660+0000","last_became_peered":"2023-03-29T14:02:17.490660+0000","last_unstale":"2023-03-29T14:05:22.331203+0000","last_undegraded":"2023-03-29T14:05:22.331203+0000","last_fullsized":"2023-03-29T14:05:22.331203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'69","reported_seq":293,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.332452+0000","last_change":"2023-03-29T14:02:17.438353+0000","last_active":"2023-03-29T14:05:22.332452+0000","last_peered":"2023-03-29T14:05:22.332452+0000","last_clean":"2023-03-29T14:05:22.332452+0000","last_became_active":"2023-03-29T14:02:17.431599+0000","last_became_peered":"2023-03-29T14:02:17.431599+0000","last_unstale":"2023-03-29T14:05:22.332452+0000","last_undegraded":"2023-03-29T14:05:22.332452+0000","last_fullsized":"2023-03-29T14:05:22.332452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193462,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":51654,"num_write":126,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":262,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.323541+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:22.323541+0000","last_peered":"2023-03-29T14:05:22.323541+0000","last_clean":"2023-03-29T14:05:22.323541+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:22.323541+0000","last_undegraded":"2023-03-29T14:05:22.323541+0000","last_fullsized":"2023-03-29T14:05:22.323541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.545414+0000","last_change":"2023-03-29T14:02:10.410007+0000","last_active":"2023-03-29T14:05:21.545414+0000","last_peered":"2023-03-29T14:05:21.545414+0000","last_clean":"2023-03-29T14:05:21.545414+0000","last_became_active":"2023-03-29T14:02:10.401517+0000","last_became_peered":"2023-03-29T14:02:10.401517+0000","last_unstale":"2023-03-29T14:05:21.545414+0000","last_undegraded":"2023-03-29T14:05:21.545414+0000","last_fullsized":"2023-03-29T14:05:21.545414+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":186,"reported_epoch":174,"state":"stale+active+clean","last_fresh":"2023-03-29T14:04:58.341297+0000","last_change":"2023-03-29T14:02:17.449695+0000","last_active":"2023-03-29T14:04:58.341297+0000","last_peered":"2023-03-29T14:04:58.341297+0000","last_clean":"2023-03-29T14:04:58.341297+0000","last_became_active":"2023-03-29T14:02:17.442704+0000","last_became_peered":"2023-03-29T14:02:17.442704+0000","last_unstale":"2023-03-29T14:05:29.318049+0000","last_undegraded":"2023-03-29T14:04:58.341297+0000","last_fullsized":"2023-03-29T14:04:58.341297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":24201,"num_write":66,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.331729+0000","last_change":"2023-03-29T14:02:10.375236+0000","last_active":"2023-03-29T14:05:22.331729+0000","last_peered":"2023-03-29T14:05:22.331729+0000","last_clean":"2023-03-29T14:05:22.331729+0000","last_became_active":"2023-03-29T14:02:10.372022+0000","last_became_peered":"2023-03-29T14:02:10.372022+0000","last_unstale":"2023-03-29T14:05:22.331729+0000","last_undegraded":"2023-03-29T14:05:22.331729+0000","last_fullsized":"2023-03-29T14:05:22.331729+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":264,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.386328+0000","last_change":"2023-03-29T14:02:17.560112+0000","last_active":"2023-03-29T14:05:21.386328+0000","last_peered":"2023-03-29T14:05:21.386328+0000","last_clean":"2023-03-29T14:05:21.386328+0000","last_became_active":"2023-03-29T14:02:17.547726+0000","last_became_peered":"2023-03-29T14:02:17.547726+0000","last_unstale":"2023-03-29T14:05:21.386328+0000","last_undegraded":"2023-03-29T14:05:21.386328+0000","last_fullsized":"2023-03-29T14:05:21.386328+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":174,"state":"stale+active+clean","last_fresh":"2023-03-29T14:04:58.343794+0000","last_change":"2023-03-29T14:02:10.333140+0000","last_active":"2023-03-29T14:04:58.343794+0000","last_peered":"2023-03-29T14:04:58.343794+0000","last_clean":"2023-03-29T14:04:58.343794+0000","last_became_active":"2023-03-29T14:02:10.328525+0000","last_became_peered":"2023-03-29T14:02:10.328525+0000","last_unstale":"2023-03-29T14:05:29.318050+0000","last_undegraded":"2023-03-29T14:04:58.343794+0000","last_fullsized":"2023-03-29T14:04:58.343794+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":290,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.542673+0000","last_change":"2023-03-29T14:02:17.474058+0000","last_active":"2023-03-29T14:05:21.542673+0000","last_peered":"2023-03-29T14:05:21.542673+0000","last_clean":"2023-03-29T14:05:21.542673+0000","last_became_active":"2023-03-29T14:02:17.460069+0000","last_became_peered":"2023-03-29T14:02:17.460069+0000","last_unstale":"2023-03-29T14:05:21.542673+0000","last_undegraded":"2023-03-29T14:05:21.542673+0000","last_fullsized":"2023-03-29T14:05:21.542673+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":143,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.325735+0000","last_change":"2023-03-29T14:02:10.316305+0000","last_active":"2023-03-29T14:05:22.325735+0000","last_peered":"2023-03-29T14:05:22.325735+0000","last_clean":"2023-03-29T14:05:22.325735+0000","last_became_active":"2023-03-29T14:02:10.313117+0000","last_became_peered":"2023-03-29T14:02:10.313117+0000","last_unstale":"2023-03-29T14:05:22.325735+0000","last_undegraded":"2023-03-29T14:05:22.325735+0000","last_fullsized":"2023-03-29T14:05:22.325735+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":365,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.506432+0000","last_change":"2023-03-29T14:02:09.259201+0000","last_active":"2023-03-29T14:05:21.506432+0000","last_peered":"2023-03-29T14:05:21.506432+0000","last_clean":"2023-03-29T14:05:21.506432+0000","last_became_active":"2023-03-29T14:02:09.253093+0000","last_became_peered":"2023-03-29T14:02:09.253093+0000","last_unstale":"2023-03-29T14:05:21.506432+0000","last_undegraded":"2023-03-29T14:05:21.506432+0000","last_fullsized":"2023-03-29T14:05:21.506432+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":316,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.324082+0000","last_change":"2023-03-29T14:02:17.471299+0000","last_active":"2023-03-29T14:05:22.324082+0000","last_peered":"2023-03-29T14:05:22.324082+0000","last_clean":"2023-03-29T14:05:22.324082+0000","last_became_active":"2023-03-29T14:02:17.464613+0000","last_became_peered":"2023-03-29T14:02:17.464613+0000","last_unstale":"2023-03-29T14:05:22.324082+0000","last_undegraded":"2023-03-29T14:05:22.324082+0000","last_fullsized":"2023-03-29T14:05:22.324082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.327413+0000","last_change":"2023-03-29T14:02:10.367291+0000","last_active":"2023-03-29T14:05:22.327413+0000","last_peered":"2023-03-29T14:05:22.327413+0000","last_clean":"2023-03-29T14:05:22.327413+0000","last_became_active":"2023-03-29T14:02:10.363801+0000","last_became_peered":"2023-03-29T14:02:10.363801+0000","last_unstale":"2023-03-29T14:05:22.327413+0000","last_undegraded":"2023-03-29T14:05:22.327413+0000","last_fullsized":"2023-03-29T14:05:22.327413+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"179'155","reported_seq":502,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.376216+0000","last_change":"2023-03-29T14:02:17.467293+0000","last_active":"2023-03-29T14:05:21.376216+0000","last_peered":"2023-03-29T14:05:21.376216+0000","last_clean":"2023-03-29T14:05:21.376216+0000","last_became_active":"2023-03-29T14:02:17.441042+0000","last_became_peered":"2023-03-29T14:02:17.441042+0000","last_unstale":"2023-03-29T14:05:21.376216+0000","last_undegraded":"2023-03-29T14:05:21.376216+0000","last_fullsized":"2023-03-29T14:05:21.376216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277926,"num_objects":58,"num_object_clones":44,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":254,"num_read_kb":159769,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.552565+0000","last_change":"2023-03-29T14:02:10.374276+0000","last_active":"2023-03-29T14:05:21.552565+0000","last_peered":"2023-03-29T14:05:21.552565+0000","last_clean":"2023-03-29T14:05:21.552565+0000","last_became_active":"2023-03-29T14:02:10.364406+0000","last_became_peered":"2023-03-29T14:02:10.364406+0000","last_unstale":"2023-03-29T14:05:21.552565+0000","last_undegraded":"2023-03-29T14:05:21.552565+0000","last_fullsized":"2023-03-29T14:05:21.552565+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":260,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.518605+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:21.518605+0000","last_peered":"2023-03-29T14:05:21.518605+0000","last_clean":"2023-03-29T14:05:21.518605+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:21.518605+0000","last_undegraded":"2023-03-29T14:05:21.518605+0000","last_fullsized":"2023-03-29T14:05:21.518605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":385,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.522366+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:21.522366+0000","last_peered":"2023-03-29T14:05:21.522366+0000","last_clean":"2023-03-29T14:05:21.522366+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:21.522366+0000","last_undegraded":"2023-03-29T14:05:21.522366+0000","last_fullsized":"2023-03-29T14:05:21.522366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":159,"num_read_kb":92204,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.550339+0000","last_change":"2023-03-29T14:02:10.353942+0000","last_active":"2023-03-29T14:05:21.550339+0000","last_peered":"2023-03-29T14:05:21.550339+0000","last_clean":"2023-03-29T14:05:21.550339+0000","last_became_active":"2023-03-29T14:02:10.328666+0000","last_became_peered":"2023-03-29T14:02:10.328666+0000","last_unstale":"2023-03-29T14:05:21.550339+0000","last_undegraded":"2023-03-29T14:05:21.550339+0000","last_fullsized":"2023-03-29T14:05:21.550339+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":143,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-03-29T14:05:22.326782+0000","last_change":"2023-03-29T14:02:10.274938+0000","last_active":"2023-03-29T14:05:22.326782+0000","last_peered":"2023-03-29T14:05:22.326782+0000","last_clean":"2023-03-29T14:05:22.326782+0000","last_became_active":"2023-03-29T14:02:10.271946+0000","last_became_peered":"2023-03-29T14:02:10.271946+0000","last_unstale":"2023-03-29T14:05:22.326782+0000","last_undegraded":"2023-03-29T14:05:22.326782+0000","last_fullsized":"2023-03-29T14:05:22.326782+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"175'97","reported_seq":356,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-03-29T14:05:21.344448+0000","last_change":"2023-03-29T14:02:17.542896+0000","last_active":"2023-03-29T14:05:21.344448+0000","last_peered":"2023-03-29T14:05:21.344448+0000","last_clean":"2023-03-29T14:05:21.344448+0000","last_became_active":"2023-03-29T14:02:17.528470+0000","last_became_peered":"2023-03-29T14:02:17.528470+0000","last_unstale":"2023-03-29T14:05:21.344448+0000","last_undegraded":"2023-03-29T14:05:21.344448+0000","last_fullsized":"2023-03-29T14:05:21.344448+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19462704,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":83787,"num_write":186,"num_write_kb":53566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":191130788,"num_objects":448,"num_object_clones":348,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":1749,"num_read_kb":1043111,"num_write":2192,"num_write_kb":637270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1115,"ondisk_log_size":1115,"up":48,"acting":48,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":225980,"kb_used_data":213108,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94145860,"statfs":{"total":96636764160,"available":96405360640,"internally_reserved":0,"allocated":218222592,"data_stored":260927951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":260556,"kb_used_data":246788,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94111284,"statfs":{"total":96636764160,"available":96369954816,"internally_reserved":0,"allocated":252710912,"data_stored":306969393,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":239672,"kb_used_data":226224,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94132168,"statfs":{"total":96636764160,"available":96391340032,"internally_reserved":0,"allocated":231653376,"data_stored":282465691,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:30.472 INFO:tasks.rados.rados.0.smithi017.stdout:695: done (6 left) 2023-03-29T14:05:30.472 INFO:tasks.rados.rados.0.smithi017.stdout:696: done (5 left) 2023-03-29T14:05:30.472 INFO:tasks.rados.rados.0.smithi017.stdout:697: done (4 left) 2023-03-29T14:05:30.472 INFO:tasks.rados.rados.0.smithi017.stdout:698: done (3 left) 2023-03-29T14:05:30.472 INFO:tasks.rados.rados.0.smithi017.stdout:700: done (2 left) 2023-03-29T14:05:30.472 INFO:tasks.rados.rados.0.smithi017.stdout:701: done (1 left) 2023-03-29T14:05:30.472 INFO:tasks.rados.rados.0.smithi017.stdout:702: snap_remove snap 53 2023-03-29T14:05:31.329 INFO:tasks.rados.rados.0.smithi017.stdout:702: done (1 left) 2023-03-29T14:05:31.329 INFO:tasks.rados.rados.0.smithi017.stdout:703: write oid 40 current snap is 82 2023-03-29T14:05:31.329 INFO:tasks.rados.rados.0.smithi017.stdout:703: seq_num 340 ranges {514380=704192,1629760=478435,2604288=442602} 2023-03-29T14:05:31.336 INFO:tasks.rados.rados.0.smithi017.stdout:703: writing smithi017107884-40 from 514380 to 1218572 tid 1 2023-03-29T14:05:31.340 INFO:tasks.rados.rados.0.smithi017.stdout:703: writing smithi017107884-40 from 1629760 to 2108195 tid 2 2023-03-29T14:05:31.346 INFO:tasks.rados.rados.0.smithi017.stdout:703: writing smithi017107884-40 from 2604288 to 3046890 tid 3 2023-03-29T14:05:31.346 INFO:tasks.rados.rados.0.smithi017.stdout:704: read oid 17 snap 79 2023-03-29T14:05:31.346 INFO:tasks.rados.rados.0.smithi017.stdout:704: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:05:31.346 INFO:tasks.rados.rados.0.smithi017.stdout:705: read oid 3 snap -1 2023-03-29T14:05:31.347 INFO:tasks.rados.rados.0.smithi017.stdout:705: expect deleted 2023-03-29T14:05:31.347 INFO:tasks.rados.rados.0.smithi017.stdout:706: read oid 13 snap 79 2023-03-29T14:05:31.347 INFO:tasks.rados.rados.0.smithi017.stdout:706: expect deleted 2023-03-29T14:05:31.347 INFO:tasks.rados.rados.0.smithi017.stdout:693: done (4 left) 2023-03-29T14:05:31.347 INFO:tasks.rados.rados.0.smithi017.stdout:707: read oid 10 snap -1 2023-03-29T14:05:31.347 INFO:tasks.rados.rados.0.smithi017.stdout:707: expect (ObjNum 306 snap 78 seq_num 306) 2023-03-29T14:05:31.347 INFO:tasks.rados.rados.0.smithi017.stdout:708: delete oid 30 current snap is 82 2023-03-29T14:05:31.393 INFO:tasks.rados.rados.0.smithi017.stdout:704: expect (ObjNum 219 snap 49 seq_num 219) 2023-03-29T14:05:31.751 INFO:tasks.rados.rados.0.smithi017.stdout:707: expect (ObjNum 306 snap 78 seq_num 306) 2023-03-29T14:05:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:31.928 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:31.929 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:32.016 INFO:tasks.rados.rados.0.smithi017.stdout:703: finishing write tid 1 to smithi017107884-40 2023-03-29T14:05:32.017 INFO:tasks.rados.rados.0.smithi017.stdout:703: finishing write tid 2 to smithi017107884-40 2023-03-29T14:05:32.017 INFO:tasks.rados.rados.0.smithi017.stdout:703: finishing write tid 3 to smithi017107884-40 2023-03-29T14:05:32.017 INFO:tasks.rados.rados.0.smithi017.stdout:703: finishing write tid 4 to smithi017107884-40 2023-03-29T14:05:32.017 INFO:tasks.rados.rados.0.smithi017.stdout:703: finishing write tid 5 to smithi017107884-40 2023-03-29T14:05:32.017 INFO:tasks.rados.rados.0.smithi017.stdout:703: oid 40 updating version 0 to 103 2023-03-29T14:05:32.017 INFO:tasks.rados.rados.0.smithi017.stdout:703: oid 40 version 103 is already newer than 103 2023-03-29T14:05:32.017 INFO:tasks.rados.rados.0.smithi017.stdout:703: oid 40 version 103 is already newer than 103 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:703: oid 40 version 103 is already newer than 103 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 103 (ObjNum 340 snap 82 seq_num 340) dirty exists 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:703: left oid 40 (ObjNum 340 snap 82 seq_num 340) 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:703: done (5 left) 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:704: done (4 left) 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:705: done (3 left) 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:706: done (2 left) 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:707: done (1 left) 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:708: done (0 left) 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:709: write oid 50 current snap is 82 2023-03-29T14:05:32.018 INFO:tasks.rados.rados.0.smithi017.stdout:709: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2023-03-29T14:05:32.022 INFO:tasks.rados.rados.0.smithi017.stdout:709: writing smithi017107884-50 from 547994 to 1002684 tid 1 2023-03-29T14:05:32.028 INFO:tasks.rados.rados.0.smithi017.stdout:709: writing smithi017107884-50 from 1746724 to 2228828 tid 2 2023-03-29T14:05:32.033 INFO:tasks.rados.rados.0.smithi017.stdout:709: writing smithi017107884-50 from 2659134 to 3124280 tid 3 2023-03-29T14:05:32.037 INFO:tasks.rados.rados.0.smithi017.stdout:709: writing smithi017107884-50 from 3605976 to 3747994 tid 4 2023-03-29T14:05:32.037 INFO:tasks.rados.rados.0.smithi017.stdout:710: snap_remove snap 79 2023-03-29T14:05:32.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~22s 2023-03-29T14:05:32.334 INFO:tasks.rados.rados.0.smithi017.stdout:710: done (1 left) 2023-03-29T14:05:32.334 INFO:tasks.rados.rados.0.smithi017.stdout:711: write (excl) oid 1 current snap is 82 2023-03-29T14:05:32.334 INFO:tasks.rados.rados.0.smithi017.stdout:709: finishing write tid 1 to smithi017107884-50 2023-03-29T14:05:32.334 INFO:tasks.rados.rados.0.smithi017.stdout:711: seq_num 343 ranges {564801=646292,1921559=742115,3202910=561891} 2023-03-29T14:05:32.340 INFO:tasks.rados.rados.0.smithi017.stdout:711: writing smithi017107884-1 from 564801 to 1211093 tid 1 2023-03-29T14:05:32.348 INFO:tasks.rados.rados.0.smithi017.stdout:711: writing smithi017107884-1 from 1921559 to 2663674 tid 2 2023-03-29T14:05:32.354 INFO:tasks.rados.rados.0.smithi017.stdout:711: writing smithi017107884-1 from 3202910 to 3764801 tid 3 2023-03-29T14:05:32.354 INFO:tasks.rados.rados.0.smithi017.stdout:712: write oid 34 current snap is 82 2023-03-29T14:05:32.354 INFO:tasks.rados.rados.0.smithi017.stdout:712: seq_num 344 ranges {581608=721541,1863688=485773,3113980=604043} 2023-03-29T14:05:32.361 INFO:tasks.rados.rados.0.smithi017.stdout:712: writing smithi017107884-34 from 581608 to 1303149 tid 1 2023-03-29T14:05:32.367 INFO:tasks.rados.rados.0.smithi017.stdout:712: writing smithi017107884-34 from 1863688 to 2349461 tid 2 2023-03-29T14:05:32.375 INFO:tasks.rados.rados.0.smithi017.stdout:712: writing smithi017107884-34 from 3113980 to 3718023 tid 3 2023-03-29T14:05:32.375 INFO:tasks.rados.rados.0.smithi017.stdout:713: read oid 9 snap -1 2023-03-29T14:05:32.375 INFO:tasks.rados.rados.0.smithi017.stdout:713: expect (ObjNum 321 snap 79 seq_num 321) 2023-03-29T14:05:32.375 INFO:tasks.rados.rados.0.smithi017.stdout:714: write (excl) oid 14 current snap is 82 2023-03-29T14:05:32.375 INFO:tasks.rados.rados.0.smithi017.stdout:709: finishing write tid 2 to smithi017107884-50 2023-03-29T14:05:32.375 INFO:tasks.rados.rados.0.smithi017.stdout:714: seq_num 345 ranges {598415=796790,1922170=745784,3141403=415315} 2023-03-29T14:05:32.382 INFO:tasks.rados.rados.0.smithi017.stdout:714: writing smithi017107884-14 from 598415 to 1395205 tid 1 2023-03-29T14:05:32.389 INFO:tasks.rados.rados.0.smithi017.stdout:714: writing smithi017107884-14 from 1922170 to 2667954 tid 2 2023-03-29T14:05:32.394 INFO:tasks.rados.rados.0.smithi017.stdout:714: writing smithi017107884-14 from 3141403 to 3556718 tid 3 2023-03-29T14:05:32.395 INFO:tasks.rados.rados.0.smithi017.stdout:715: write (excl) oid 43 current snap is 82 2023-03-29T14:05:32.395 INFO:tasks.rados.rados.0.smithi017.stdout:709: finishing write tid 3 to smithi017107884-50 2023-03-29T14:05:32.395 INFO:tasks.rados.rados.0.smithi017.stdout:715: seq_num 346 ranges {615222=472039,1580652=489442,2768826=626587} 2023-03-29T14:05:32.400 INFO:tasks.rados.rados.0.smithi017.stdout:715: writing smithi017107884-43 from 615222 to 1087261 tid 1 2023-03-29T14:05:32.405 INFO:tasks.rados.rados.0.smithi017.stdout:715: writing smithi017107884-43 from 1580652 to 2070094 tid 2 2023-03-29T14:05:32.413 INFO:tasks.rados.rados.0.smithi017.stdout:715: writing smithi017107884-43 from 2768826 to 3395413 tid 3 2023-03-29T14:05:32.413 INFO:tasks.rados.rados.0.smithi017.stdout:716: write oid 17 current snap is 82 2023-03-29T14:05:32.413 INFO:tasks.rados.rados.0.smithi017.stdout:716: seq_num 347 ranges {632029=547288,1639134=749453,2796249=437859,3653524=178505} 2023-03-29T14:05:32.419 INFO:tasks.rados.rados.0.smithi017.stdout:716: writing smithi017107884-17 from 632029 to 1179317 tid 1 2023-03-29T14:05:32.426 INFO:tasks.rados.rados.0.smithi017.stdout:716: writing smithi017107884-17 from 1639134 to 2388587 tid 2 2023-03-29T14:05:32.431 INFO:tasks.rados.rados.0.smithi017.stdout:716: writing smithi017107884-17 from 2796249 to 3234108 tid 3 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:716: writing smithi017107884-17 from 3653524 to 3832029 tid 4 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:717: delete oid 49 current snap is 82 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:709: finishing write tid 4 to smithi017107884-50 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:709: finishing write tid 5 to smithi017107884-50 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:709: finishing write tid 6 to smithi017107884-50 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:709: oid 50 updating version 0 to 73 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:709: oid 50 version 73 is already newer than 70 2023-03-29T14:05:32.435 INFO:tasks.rados.rados.0.smithi017.stdout:709: oid 50 version 73 is already newer than 73 2023-03-29T14:05:32.436 INFO:tasks.rados.rados.0.smithi017.stdout:709: oid 50 version 73 is already newer than 72 2023-03-29T14:05:32.437 INFO:tasks.rados.rados.0.smithi017.stdout:709: oid 50 version 73 is already newer than 72 2023-03-29T14:05:32.437 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 73 (ObjNum 342 snap 82 seq_num 342) dirty exists 2023-03-29T14:05:32.437 INFO:tasks.rados.rados.0.smithi017.stdout:709: left oid 50 (ObjNum 342 snap 82 seq_num 342) 2023-03-29T14:05:32.437 INFO:tasks.rados.rados.0.smithi017.stdout:713: expect (ObjNum 321 snap 79 seq_num 321) 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:709: done (7 left) 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:713: done (6 left) 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:717: done (5 left) 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:718: read oid 21 snap -1 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:718: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:719: read oid 23 snap -1 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:719: expect (ObjNum 332 snap 81 seq_num 332) 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:720: snap_create 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:711: finishing write tid 1 to smithi017107884-1 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:714: finishing write tid 1 to smithi017107884-14 2023-03-29T14:05:32.791 INFO:tasks.rados.rados.0.smithi017.stdout:716: finishing write tid 1 to smithi017107884-17 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:716: finishing write tid 2 to smithi017107884-17 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: finishing write tid 2 to smithi017107884-1 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:716: finishing write tid 3 to smithi017107884-17 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:716: finishing write tid 4 to smithi017107884-17 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:712: finishing write tid 1 to smithi017107884-34 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:712: finishing write tid 2 to smithi017107884-34 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: finishing write tid 3 to smithi017107884-1 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:712: finishing write tid 3 to smithi017107884-34 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: finishing write tid 4 to smithi017107884-1 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: finishing write tid 5 to smithi017107884-1 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: oid 1 updating version 0 to 101 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: oid 1 version 101 is already newer than 101 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: oid 1 version 101 is already newer than 101 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: oid 1 version 101 is already newer than 98 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 101 (ObjNum 343 snap 82 seq_num 343) dirty exists 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:711: left oid 1 (ObjNum 343 snap 82 seq_num 343) 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:715: finishing write tid 1 to smithi017107884-43 2023-03-29T14:05:32.792 INFO:tasks.rados.rados.0.smithi017.stdout:716: finishing write tid 5 to smithi017107884-17 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:716: finishing write tid 6 to smithi017107884-17 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:716: oid 17 updating version 0 to 104 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:716: oid 17 version 104 is already newer than 104 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:716: oid 17 version 104 is already newer than 104 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:716: oid 17 version 104 is already newer than 104 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:716: oid 17 version 104 is already newer than 104 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 104 (ObjNum 347 snap 82 seq_num 347) dirty exists 2023-03-29T14:05:32.793 INFO:tasks.rados.rados.0.smithi017.stdout:716: left oid 17 (ObjNum 347 snap 82 seq_num 347) 2023-03-29T14:05:32.799 INFO:tasks.rados.rados.0.smithi017.stdout:718: expect (ObjNum 20 snap 0 seq_num 20) 2023-03-29T14:05:32.953 INFO:tasks.rados.rados.0.smithi017.stdout:715: finishing write tid 2 to smithi017107884-43 2023-03-29T14:05:32.953 INFO:tasks.rados.rados.0.smithi017.stdout:714: finishing write tid 2 to smithi017107884-14 2023-03-29T14:05:32.953 INFO:tasks.rados.rados.0.smithi017.stdout:715: finishing write tid 3 to smithi017107884-43 2023-03-29T14:05:32.953 INFO:tasks.rados.rados.0.smithi017.stdout:712: finishing write tid 4 to smithi017107884-34 2023-03-29T14:05:32.953 INFO:tasks.rados.rados.0.smithi017.stdout:715: finishing write tid 4 to smithi017107884-43 2023-03-29T14:05:32.953 INFO:tasks.rados.rados.0.smithi017.stdout:712: finishing write tid 5 to smithi017107884-34 2023-03-29T14:05:32.953 INFO:tasks.rados.rados.0.smithi017.stdout:712: oid 34 updating version 0 to 99 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:712: oid 34 version 99 is already newer than 99 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:712: oid 34 version 99 is already newer than 99 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:712: oid 34 version 99 is already newer than 99 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 99 (ObjNum 344 snap 82 seq_num 344) dirty exists 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:712: left oid 34 (ObjNum 344 snap 82 seq_num 344) 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:715: finishing write tid 5 to smithi017107884-43 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:715: oid 43 updating version 0 to 77 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:715: oid 43 version 77 is already newer than 74 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:715: oid 43 version 77 is already newer than 77 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:715: oid 43 version 77 is already newer than 77 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 77 (ObjNum 346 snap 82 seq_num 346) dirty exists 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:715: left oid 43 (ObjNum 346 snap 82 seq_num 346) 2023-03-29T14:05:32.954 INFO:tasks.rados.rados.0.smithi017.stdout:719: expect (ObjNum 332 snap 81 seq_num 332) 2023-03-29T14:05:33.202 INFO:tasks.rados.rados.0.smithi017.stdout:714: finishing write tid 3 to smithi017107884-14 2023-03-29T14:05:33.202 INFO:tasks.rados.rados.0.smithi017.stdout:714: finishing write tid 4 to smithi017107884-14 2023-03-29T14:05:33.202 INFO:tasks.rados.rados.0.smithi017.stdout:714: finishing write tid 5 to smithi017107884-14 2023-03-29T14:05:33.202 INFO:tasks.rados.rados.0.smithi017.stdout:714: oid 14 updating version 0 to 70 2023-03-29T14:05:33.202 INFO:tasks.rados.rados.0.smithi017.stdout:714: oid 14 updating version 70 to 73 2023-03-29T14:05:33.203 INFO:tasks.rados.rados.0.smithi017.stdout:714: oid 14 version 73 is already newer than 73 2023-03-29T14:05:33.203 INFO:tasks.rados.rados.0.smithi017.stdout:714: oid 14 version 73 is already newer than 73 2023-03-29T14:05:33.203 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 73 (ObjNum 345 snap 82 seq_num 345) dirty exists 2023-03-29T14:05:33.203 INFO:tasks.rados.rados.0.smithi017.stdout:714: left oid 14 (ObjNum 345 snap 82 seq_num 345) 2023-03-29T14:05:33.336 INFO:tasks.rados.rados.0.smithi017.stdout:711: done (7 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:712: done (6 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:714: done (5 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:715: done (4 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:716: done (3 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:718: done (2 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:719: done (1 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:720: done (0 left) 2023-03-29T14:05:33.337 INFO:tasks.rados.rados.0.smithi017.stdout:721: read oid 15 snap 82 2023-03-29T14:05:33.338 INFO:tasks.rados.rados.0.smithi017.stdout:721: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:05:33.338 INFO:tasks.rados.rados.0.smithi017.stdout:722: read oid 48 snap 82 2023-03-29T14:05:33.338 INFO:tasks.rados.rados.0.smithi017.stdout:722: expect deleted 2023-03-29T14:05:33.338 INFO:tasks.rados.rados.0.smithi017.stdout:723: snap_create 2023-03-29T14:05:33.415 INFO:tasks.rados.rados.0.smithi017.stdout:721: expect (ObjNum 196 snap 40 seq_num 196) 2023-03-29T14:05:33.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:34.058+0000 7f62c31e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:34.063+0000 7f62c31e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:34.126 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:34.129 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:34.129 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:34.341 INFO:tasks.rados.rados.0.smithi017.stdout:721: done (2 left) 2023-03-29T14:05:34.341 INFO:tasks.rados.rados.0.smithi017.stdout:722: done (1 left) 2023-03-29T14:05:34.341 INFO:tasks.rados.rados.0.smithi017.stdout:723: done (0 left) 2023-03-29T14:05:34.341 INFO:tasks.rados.rados.0.smithi017.stdout:724: snap_create 2023-03-29T14:05:34.652 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:34.652 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:34.652 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:34.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:34.779+0000 7fd3a7dbb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:34.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:34.786+0000 7fd3a7dbb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:34.988 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:34.991 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:35.002 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2023-03-29T14:05:34.339397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":188378620,"num_objects":418,"num_object_clones":322,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":1736,"num_read_kb":943433,"num_write":2334,"num_write_kb":604732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1216,"ondisk_log_size":1216,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":882376,"kb_used_data":829992,"kb_used_omap":0,"kb_used_meta":52352,"kb_avail":376604984,"statfs":{"total":386547056640,"available":385643503616,"internally_reserved":0,"allocated":849911808,"data_stored":1023624804,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":53608448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3932904,"num_objects":-34,"num_object_clones":-26,"num_object_copies":-102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-6,"num_read":-155,"num_read_kb":-99798,"num_write":-132,"num_write_kb":-36326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015095"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":10,"reported_epoch":187,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.459869+0000","last_change":"2023-03-29T14:05:30.457299+0000","last_active":"2023-03-29T14:05:30.459869+0000","last_peered":"2023-03-29T14:05:30.459869+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:05:30.459869+0000","last_undegraded":"2023-03-29T14:05:30.459869+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":186,"reported_epoch":186,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.558443+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:05:30.558443+0000","last_peered":"2023-03-29T14:05:30.558443+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:05:30.558443+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"174'99","reported_seq":367,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-03-29T14:05:31.396304+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:31.396304+0000","last_peered":"2023-03-29T14:05:31.396304+0000","last_clean":"2023-03-29T14:05:31.396304+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:31.396304+0000","last_undegraded":"2023-03-29T14:05:31.396304+0000","last_fullsized":"2023-03-29T14:05:31.396304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647350,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":141,"num_read_kb":99174,"num_write":190,"num_write_kb":56432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"187'103","reported_seq":379,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-03-29T14:05:31.374145+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:31.374145+0000","last_peered":"2023-03-29T14:05:31.374145+0000","last_clean":"2023-03-29T14:05:31.374145+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:31.374145+0000","last_undegraded":"2023-03-29T14:05:31.374145+0000","last_fullsized":"2023-03-29T14:05:31.374145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14218854,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":161,"num_read_kb":95713,"num_write":198,"num_write_kb":55758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"175'95","reported_seq":355,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-03-29T14:05:29.414695+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:29.414695+0000","last_peered":"2023-03-29T14:05:29.414695+0000","last_clean":"2023-03-29T14:05:29.414695+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:29.414695+0000","last_undegraded":"2023-03-29T14:05:29.414695+0000","last_fullsized":"2023-03-29T14:05:29.414695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790556,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":71228,"num_write":194,"num_write_kb":57986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":148,"reported_epoch":187,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.479841+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:05:30.479841+0000","last_peered":"2023-03-29T14:05:30.479841+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:05:30.479841+0000","last_undegraded":"2023-03-29T14:05:30.479841+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'69","reported_seq":301,"reported_epoch":187,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.509131+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:05:30.509131+0000","last_peered":"2023-03-29T14:05:30.509131+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:05:30.509131+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193462,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":103,"num_read_kb":51654,"num_write":126,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":265,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-03-29T14:05:30.338598+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:30.338598+0000","last_peered":"2023-03-29T14:05:30.338598+0000","last_clean":"2023-03-29T14:05:30.338598+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:30.338598+0000","last_undegraded":"2023-03-29T14:05:30.338598+0000","last_fullsized":"2023-03-29T14:05:30.338598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":186,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.529147+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:05:30.529147+0000","last_peered":"2023-03-29T14:05:30.529147+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:05:30.529147+0000","last_undegraded":"2023-03-29T14:05:30.529147+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":9,"reported_epoch":186,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.481510+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:05:30.481510+0000","last_peered":"2023-03-29T14:05:30.481510+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:05:30.481510+0000","last_undegraded":"2023-03-29T14:05:30.481510+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":151,"reported_epoch":187,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.487892+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:05:30.487892+0000","last_peered":"2023-03-29T14:05:30.487892+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:05:30.487892+0000","last_undegraded":"2023-03-29T14:05:30.487892+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":272,"reported_epoch":186,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.556910+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:05:30.556910+0000","last_peered":"2023-03-29T14:05:30.556910+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:05:30.556910+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":186,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.544084+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:05:30.544084+0000","last_peered":"2023-03-29T14:05:30.544084+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:05:30.544084+0000","last_undegraded":"2023-03-29T14:05:30.544084+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"161'67","reported_seq":298,"reported_epoch":186,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.631325+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:05:30.631325+0000","last_peered":"2023-03-29T14:05:30.631325+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:05:30.631325+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20773518,"num_objects":26,"num_object_clones":20,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":45589,"num_write":140,"num_write_kb":43182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":26}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":187,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.525051+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:05:30.525051+0000","last_peered":"2023-03-29T14:05:30.525051+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:05:30.525051+0000","last_undegraded":"2023-03-29T14:05:30.525051+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":373,"reported_epoch":186,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.600308+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:05:30.600308+0000","last_peered":"2023-03-29T14:05:30.600308+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:05:30.600308+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":324,"reported_epoch":187,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.474466+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:05:30.474466+0000","last_peered":"2023-03-29T14:05:30.474466+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:05:30.474466+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":89914,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":151,"reported_epoch":187,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.471317+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:05:30.471317+0000","last_peered":"2023-03-29T14:05:30.471317+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:05:30.471317+0000","last_undegraded":"2023-03-29T14:05:30.471317+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"179'155","reported_seq":510,"reported_epoch":186,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.462876+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:05:30.462876+0000","last_peered":"2023-03-29T14:05:30.462876+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:05:30.462876+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277926,"num_objects":58,"num_object_clones":44,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":254,"num_read_kb":159769,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":58}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":186,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.579504+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:05:30.579504+0000","last_peered":"2023-03-29T14:05:30.579504+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:05:30.579504+0000","last_undegraded":"2023-03-29T14:05:30.579504+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":263,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-03-29T14:05:29.410480+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:29.410480+0000","last_peered":"2023-03-29T14:05:29.410480+0000","last_clean":"2023-03-29T14:05:29.410480+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:29.410480+0000","last_undegraded":"2023-03-29T14:05:29.410480+0000","last_fullsized":"2023-03-29T14:05:29.410480+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":388,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-03-29T14:05:29.437827+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:29.437827+0000","last_peered":"2023-03-29T14:05:29.437827+0000","last_clean":"2023-03-29T14:05:29.437827+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:29.437827+0000","last_undegraded":"2023-03-29T14:05:29.437827+0000","last_fullsized":"2023-03-29T14:05:29.437827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":159,"num_read_kb":92204,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":186,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.611641+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:05:30.611641+0000","last_peered":"2023-03-29T14:05:30.611641+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:05:30.611641+0000","last_undegraded":"2023-03-29T14:05:30.611641+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":187,"state":"active+undersized","last_fresh":"2023-03-29T14:05:30.514062+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:05:30.514062+0000","last_peered":"2023-03-29T14:05:30.514062+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:05:30.514062+0000","last_undegraded":"2023-03-29T14:05:30.514062+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"175'97","reported_seq":364,"reported_epoch":186,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:30.541698+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:05:30.541698+0000","last_peered":"2023-03-29T14:05:30.541698+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:05:30.541698+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19462704,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":83787,"num_write":186,"num_write_kb":53566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":187197884,"num_objects":414,"num_object_clones":322,"num_object_copies":1242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":1594,"num_read_kb":943313,"num_write":2060,"num_write_kb":600944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1119,"ondisk_log_size":1119,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":226052,"kb_used_data":213116,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94145788,"statfs":{"total":96636764160,"available":96405286912,"internally_reserved":0,"allocated":218230784,"data_stored":260935337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":260632,"kb_used_data":246800,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94111208,"statfs":{"total":96636764160,"available":96369876992,"internally_reserved":0,"allocated":252723200,"data_stored":306980472,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":239748,"kb_used_data":226236,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94132092,"statfs":{"total":96636764160,"available":96391262208,"internally_reserved":0,"allocated":231665664,"data_stored":282476770,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:35.006 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.9 3.3 1.0 3.1 3.6 2023-03-29T14:05:35.143 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:35.132+0000 7f932d5df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:35.146 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:35.145+0000 7f932d5df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:35.342 INFO:tasks.rados.rados.0.smithi017.stdout:724: done (0 left) 2023-03-29T14:05:35.342 INFO:tasks.rados.rados.0.smithi017.stdout:725: delete oid 15 current snap is 85 2023-03-29T14:05:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3,3.6] on osd.0 to force-backfill; instructing pg(s) [1.0,3.e] on osd.1 to force-backfill; instructing pg(s) [3.9] on osd.3 to force-backfill; 2023-03-29T14:05:35.355 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:05:35.454 INFO:tasks.rados.rados.0.smithi017.stdout:725: done (0 left) 2023-03-29T14:05:35.455 INFO:tasks.rados.rados.0.smithi017.stdout:726: write oid 18 current snap is 85 2023-03-29T14:05:35.455 INFO:tasks.rados.rados.0.smithi017.stdout:726: seq_num 350 ranges {682450=489388,1814580=496780,2878518=788028} 2023-03-29T14:05:35.463 INFO:tasks.rados.rados.0.smithi017.stdout:726: writing smithi017107884-18 from 682450 to 1171838 tid 1 2023-03-29T14:05:35.471 INFO:tasks.rados.rados.0.smithi017.stdout:726: writing smithi017107884-18 from 1814580 to 2311360 tid 2 2023-03-29T14:05:35.481 INFO:tasks.rados.rados.0.smithi017.stdout:726: writing smithi017107884-18 from 2878518 to 3666546 tid 3 2023-03-29T14:05:35.482 INFO:tasks.rados.rados.0.smithi017.stdout:727: snap_create 2023-03-29T14:05:35.520 INFO:tasks.rados.rados.0.smithi017.stdout:726: finishing write tid 1 to smithi017107884-18 2023-03-29T14:05:35.530 INFO:tasks.rados.rados.0.smithi017.stdout:726: finishing write tid 2 to smithi017107884-18 2023-03-29T14:05:35.573 INFO:tasks.rados.rados.0.smithi017.stdout:726: finishing write tid 3 to smithi017107884-18 2023-03-29T14:05:35.573 INFO:tasks.rados.rados.0.smithi017.stdout:726: finishing write tid 4 to smithi017107884-18 2023-03-29T14:05:35.574 INFO:tasks.rados.rados.0.smithi017.stdout:726: finishing write tid 5 to smithi017107884-18 2023-03-29T14:05:35.574 INFO:tasks.rados.rados.0.smithi017.stdout:726: oid 18 updating version 0 to 78 2023-03-29T14:05:35.575 INFO:tasks.rados.rados.0.smithi017.stdout:726: oid 18 version 78 is already newer than 78 2023-03-29T14:05:35.575 INFO:tasks.rados.rados.0.smithi017.stdout:726: oid 18 version 78 is already newer than 78 2023-03-29T14:05:35.575 INFO:tasks.rados.rados.0.smithi017.stdout:726: oid 18 version 78 is already newer than 78 2023-03-29T14:05:35.575 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 78 (ObjNum 350 snap 85 seq_num 350) dirty exists 2023-03-29T14:05:35.575 INFO:tasks.rados.rados.0.smithi017.stdout:726: left oid 18 (ObjNum 350 snap 85 seq_num 350) 2023-03-29T14:05:36.130 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:36.260 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:36.258+0000 7fa100ba6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:36.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:36.262+0000 7fa100ba6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:36.325 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:36.328 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:36.328 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:36.348 INFO:tasks.rados.rados.0.smithi017.stdout:726: done (1 left) 2023-03-29T14:05:36.349 INFO:tasks.rados.rados.0.smithi017.stdout:727: done (0 left) 2023-03-29T14:05:36.349 INFO:tasks.rados.rados.0.smithi017.stdout:728: write (excl) oid 19 current snap is 86 2023-03-29T14:05:36.349 INFO:tasks.rados.rados.0.smithi017.stdout:728: seq_num 351 ranges {699257=564637,1873062=640438,3305941=593316} 2023-03-29T14:05:36.355 INFO:tasks.rados.rados.0.smithi017.stdout:728: writing smithi017107884-19 from 699257 to 1263894 tid 1 2023-03-29T14:05:36.362 INFO:tasks.rados.rados.0.smithi017.stdout:728: writing smithi017107884-19 from 1873062 to 2513500 tid 2 2023-03-29T14:05:36.370 INFO:tasks.rados.rados.0.smithi017.stdout:728: writing smithi017107884-19 from 3305941 to 3899257 tid 3 2023-03-29T14:05:36.370 INFO:tasks.rados.rados.0.smithi017.stdout:729: write oid 9 current snap is 86 2023-03-29T14:05:36.370 INFO:tasks.rados.rados.0.smithi017.stdout:729: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2023-03-29T14:05:36.377 INFO:tasks.rados.rados.0.smithi017.stdout:729: writing smithi017107884-9 from 716064 to 1355950 tid 1 2023-03-29T14:05:36.383 INFO:tasks.rados.rados.0.smithi017.stdout:729: writing smithi017107884-9 from 1815191 to 2315640 tid 2 2023-03-29T14:05:36.389 INFO:tasks.rados.rados.0.smithi017.stdout:729: writing smithi017107884-9 from 2817011 to 3343936 tid 3 2023-03-29T14:05:36.389 INFO:tasks.rados.rados.0.smithi017.stdout:730: delete oid 5 current snap is 86 2023-03-29T14:05:36.532 INFO:tasks.rados.rados.0.smithi017.stdout:730: done (2 left) 2023-03-29T14:05:36.532 INFO:tasks.rados.rados.0.smithi017.stdout:731: snap_remove snap 82 2023-03-29T14:05:37.352 INFO:tasks.rados.rados.0.smithi017.stdout:731: done (2 left) 2023-03-29T14:05:37.352 INFO:tasks.rados.rados.0.smithi017.stdout:732: read oid 6 snap -1 2023-03-29T14:05:37.352 INFO:tasks.rados.rados.0.smithi017.stdout:732: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:05:37.352 INFO:tasks.rados.rados.0.smithi017.stdout:733: snap_remove snap 85 2023-03-29T14:05:37.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~28s 2023-03-29T14:05:38.211 INFO:tasks.rados.rados.0.smithi017.stdout:733: done (3 left) 2023-03-29T14:05:38.211 INFO:tasks.rados.rados.0.smithi017.stdout:734: delete oid 2 current snap is 86 2023-03-29T14:05:38.211 INFO:tasks.rados.rados.0.smithi017.stdout:729: finishing write tid 1 to smithi017107884-9 2023-03-29T14:05:38.211 INFO:tasks.rados.rados.0.smithi017.stdout:729: finishing write tid 2 to smithi017107884-9 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: finishing write tid 3 to smithi017107884-9 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: finishing write tid 4 to smithi017107884-9 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: finishing write tid 5 to smithi017107884-9 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: oid 9 updating version 0 to 108 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: oid 9 version 108 is already newer than 108 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: oid 9 version 108 is already newer than 108 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: oid 9 version 108 is already newer than 108 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 108 (ObjNum 352 snap 86 seq_num 352) dirty exists 2023-03-29T14:05:38.212 INFO:tasks.rados.rados.0.smithi017.stdout:729: left oid 9 (ObjNum 352 snap 86 seq_num 352) 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: finishing write tid 1 to smithi017107884-19 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: finishing write tid 2 to smithi017107884-19 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: finishing write tid 3 to smithi017107884-19 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: finishing write tid 4 to smithi017107884-19 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: finishing write tid 5 to smithi017107884-19 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: oid 19 updating version 0 to 106 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: oid 19 version 106 is already newer than 106 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: oid 19 version 106 is already newer than 106 2023-03-29T14:05:38.213 INFO:tasks.rados.rados.0.smithi017.stdout:728: oid 19 version 106 is already newer than 102 2023-03-29T14:05:38.214 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 106 (ObjNum 351 snap 86 seq_num 351) dirty exists 2023-03-29T14:05:38.214 INFO:tasks.rados.rados.0.smithi017.stdout:728: left oid 19 (ObjNum 351 snap 86 seq_num 351) 2023-03-29T14:05:38.214 INFO:tasks.rados.rados.0.smithi017.stdout:732: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:05:38.329 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:38.428 INFO:tasks.rados.rados.0.smithi017.stdout:728: done (3 left) 2023-03-29T14:05:38.428 INFO:tasks.rados.rados.0.smithi017.stdout:729: done (2 left) 2023-03-29T14:05:38.428 INFO:tasks.rados.rados.0.smithi017.stdout:732: done (1 left) 2023-03-29T14:05:38.428 INFO:tasks.rados.rados.0.smithi017.stdout:734: done (0 left) 2023-03-29T14:05:38.428 INFO:tasks.rados.rados.0.smithi017.stdout:735: snap_create 2023-03-29T14:05:38.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:38.456+0000 7f72b4332700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:38.462 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:38.461+0000 7f72b4332700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:38.610 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:38.612 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:38.613 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:39.359 INFO:tasks.rados.rados.0.smithi017.stdout:735: done (0 left) 2023-03-29T14:05:39.359 INFO:tasks.rados.rados.0.smithi017.stdout:736: write oid 12 current snap is 87 2023-03-29T14:05:39.359 INFO:tasks.rados.rados.0.smithi017.stdout:736: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2023-03-29T14:05:39.365 INFO:tasks.rados.rados.0.smithi017.stdout:736: writing smithi017107884-12 from 766485 to 1232118 tid 1 2023-03-29T14:05:39.373 INFO:tasks.rados.rados.0.smithi017.stdout:736: writing smithi017107884-12 from 1990637 to 2638413 tid 2 2023-03-29T14:05:39.380 INFO:tasks.rados.rados.0.smithi017.stdout:736: writing smithi017107884-12 from 3299280 to 3966485 tid 3 2023-03-29T14:05:39.381 INFO:tasks.rados.rados.0.smithi017.stdout:737: snap_remove snap 84 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:737: done (1 left) 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:738: delete oid 31 current snap is 87 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: finishing write tid 1 to smithi017107884-12 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: finishing write tid 2 to smithi017107884-12 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: finishing write tid 3 to smithi017107884-12 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: finishing write tid 4 to smithi017107884-12 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: finishing write tid 5 to smithi017107884-12 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: oid 12 updating version 0 to 118 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: oid 12 version 118 is already newer than 118 2023-03-29T14:05:40.363 INFO:tasks.rados.rados.0.smithi017.stdout:736: oid 12 version 118 is already newer than 117 2023-03-29T14:05:40.364 INFO:tasks.rados.rados.0.smithi017.stdout:736: oid 12 version 118 is already newer than 118 2023-03-29T14:05:40.364 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 118 (ObjNum 355 snap 87 seq_num 355) dirty exists 2023-03-29T14:05:40.364 INFO:tasks.rados.rados.0.smithi017.stdout:736: left oid 12 (ObjNum 355 snap 87 seq_num 355) 2023-03-29T14:05:40.368 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:40.368 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:40.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:40.390 INFO:tasks.rados.rados.0.smithi017.stdout:736: done (1 left) 2023-03-29T14:05:40.390 INFO:tasks.rados.rados.0.smithi017.stdout:738: done (0 left) 2023-03-29T14:05:40.390 INFO:tasks.rados.rados.0.smithi017.stdout:739: write (excl) oid 50 current snap is 87 2023-03-29T14:05:40.390 INFO:tasks.rados.rados.0.smithi017.stdout:739: seq_num 357 ranges {400099=616131,1591248=408851} 2023-03-29T14:05:40.395 INFO:tasks.rados.rados.0.smithi017.stdout:739: writing smithi017107884-50 from 400099 to 1016230 tid 1 2023-03-29T14:05:40.401 INFO:tasks.rados.rados.0.smithi017.stdout:739: writing smithi017107884-50 from 1591248 to 2000099 tid 2 2023-03-29T14:05:40.401 INFO:tasks.rados.rados.0.smithi017.stdout:740: snap_create 2023-03-29T14:05:40.504 INFO:tasks.rados.rados.0.smithi017.stdout:739: finishing write tid 1 to smithi017107884-50 2023-03-29T14:05:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:40.500+0000 7fb4c8c1f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:40.508+0000 7fb4c8c1f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:40.561 INFO:tasks.rados.rados.0.smithi017.stdout:739: finishing write tid 2 to smithi017107884-50 2023-03-29T14:05:40.561 INFO:tasks.rados.rados.0.smithi017.stdout:739: finishing write tid 3 to smithi017107884-50 2023-03-29T14:05:40.564 INFO:tasks.rados.rados.0.smithi017.stdout:739: finishing write tid 4 to smithi017107884-50 2023-03-29T14:05:40.564 INFO:tasks.rados.rados.0.smithi017.stdout:739: oid 50 updating version 0 to 78 2023-03-29T14:05:40.564 INFO:tasks.rados.rados.0.smithi017.stdout:739: oid 50 updating version 78 to 81 2023-03-29T14:05:40.564 INFO:tasks.rados.rados.0.smithi017.stdout:739: oid 50 version 81 is already newer than 81 2023-03-29T14:05:40.565 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 81 (ObjNum 357 snap 87 seq_num 357) dirty exists 2023-03-29T14:05:40.565 INFO:tasks.rados.rados.0.smithi017.stdout:739: left oid 50 (ObjNum 357 snap 87 seq_num 357) 2023-03-29T14:05:40.614 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:40.717 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:40.719 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:40.731 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2023-03-29T14:05:40.361624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197017616,"num_objects":426,"num_object_clones":330,"num_object_copies":1278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":1778,"num_read_kb":971385,"num_write":2418,"num_write_kb":632500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1252,"ondisk_log_size":1252,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":903684,"kb_used_data":850788,"kb_used_omap":0,"kb_used_meta":52864,"kb_avail":376583676,"statfs":{"total":386547056640,"available":385621684224,"internally_reserved":0,"allocated":871206912,"data_stored":1046116465,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":54132736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9512894,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":19,"num_read_kb":13172,"num_write":42,"num_write_kb":14294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014341"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":15,"reported_epoch":192,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.477082+0000","last_change":"2023-03-29T14:05:30.457299+0000","last_active":"2023-03-29T14:05:35.477082+0000","last_peered":"2023-03-29T14:05:35.477082+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:05:35.477082+0000","last_undegraded":"2023-03-29T14:05:35.477082+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":190,"reported_epoch":191,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.292236+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:05:35.292236+0000","last_peered":"2023-03-29T14:05:35.292236+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:05:35.292236+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"188'104","reported_seq":383,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-03-29T14:05:35.478453+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:35.478453+0000","last_peered":"2023-03-29T14:05:35.478453+0000","last_clean":"2023-03-29T14:05:35.478453+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:35.478453+0000","last_undegraded":"2023-03-29T14:05:35.478453+0000","last_fullsized":"2023-03-29T14:05:35.478453+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13949942,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":142,"num_read_kb":99175,"num_write":202,"num_write_kb":60172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"192'107","reported_seq":395,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-03-29T14:05:36.438414+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:36.438414+0000","last_peered":"2023-03-29T14:05:36.438414+0000","last_clean":"2023-03-29T14:05:36.438414+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:36.438414+0000","last_undegraded":"2023-03-29T14:05:36.438414+0000","last_fullsized":"2023-03-29T14:05:36.438414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15260888,"num_objects":48,"num_object_clones":40,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":168,"num_read_kb":104932,"num_write":208,"num_write_kb":59016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"188'99","reported_seq":368,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-03-29T14:05:35.246905+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:35.246905+0000","last_peered":"2023-03-29T14:05:35.246905+0000","last_clean":"2023-03-29T14:05:35.246905+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:35.246905+0000","last_undegraded":"2023-03-29T14:05:35.246905+0000","last_fullsized":"2023-03-29T14:05:35.246905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185570,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":71229,"num_write":204,"num_write_kb":61526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":153,"reported_epoch":192,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.492543+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:05:35.492543+0000","last_peered":"2023-03-29T14:05:35.492543+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:05:35.492543+0000","last_undegraded":"2023-03-29T14:05:35.492543+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"191'78","reported_seq":324,"reported_epoch":192,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.574937+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:05:35.574937+0000","last_peered":"2023-03-29T14:05:35.574937+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:05:35.574937+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15361730,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":107,"num_read_kb":51656,"num_write":146,"num_write_kb":43166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":32}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":270,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-03-29T14:05:35.489189+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:35.489189+0000","last_peered":"2023-03-29T14:05:35.489189+0000","last_clean":"2023-03-29T14:05:35.489189+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:35.489189+0000","last_undegraded":"2023-03-29T14:05:35.489189+0000","last_fullsized":"2023-03-29T14:05:35.489189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":191,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.254353+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:05:35.254353+0000","last_peered":"2023-03-29T14:05:35.254353+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:05:35.254353+0000","last_undegraded":"2023-03-29T14:05:35.254353+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":15,"reported_epoch":191,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.232043+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:05:35.232043+0000","last_peered":"2023-03-29T14:05:35.232043+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:05:35.232043+0000","last_undegraded":"2023-03-29T14:05:35.232043+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":192,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.489832+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:05:35.489832+0000","last_peered":"2023-03-29T14:05:35.489832+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:05:35.489832+0000","last_undegraded":"2023-03-29T14:05:35.489832+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":277,"reported_epoch":191,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.234994+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:05:35.234994+0000","last_peered":"2023-03-29T14:05:35.234994+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:05:35.234994+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":191,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.283408+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:05:35.283408+0000","last_peered":"2023-03-29T14:05:35.283408+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:05:35.283408+0000","last_undegraded":"2023-03-29T14:05:35.283408+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"188'77","reported_seq":322,"reported_epoch":191,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.261705+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:05:35.261705+0000","last_peered":"2023-03-29T14:05:35.261705+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:05:35.261705+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22689582,"num_objects":28,"num_object_clones":22,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":45591,"num_write":162,"num_write_kb":49304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":28}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":192,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.478976+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:05:35.478976+0000","last_peered":"2023-03-29T14:05:35.478976+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:05:35.478976+0000","last_undegraded":"2023-03-29T14:05:35.478976+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":377,"reported_epoch":191,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.295497+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:05:35.295497+0000","last_peered":"2023-03-29T14:05:35.295497+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:05:35.295497+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":332,"reported_epoch":192,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.485353+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:05:35.485353+0000","last_peered":"2023-03-29T14:05:35.485353+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:05:35.485353+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":153,"num_read_kb":93864,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":192,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.491800+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:05:35.491800+0000","last_peered":"2023-03-29T14:05:35.491800+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:05:35.491800+0000","last_undegraded":"2023-03-29T14:05:35.491800+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"179'155","reported_seq":524,"reported_epoch":191,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.407792+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:05:35.407792+0000","last_peered":"2023-03-29T14:05:35.407792+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:05:35.407792+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18689582,"num_objects":60,"num_object_clones":46,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":269,"num_read_kb":174545,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":191,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.252491+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:05:35.252491+0000","last_peered":"2023-03-29T14:05:35.252491+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:05:35.252491+0000","last_undegraded":"2023-03-29T14:05:35.252491+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":267,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-03-29T14:05:35.258381+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:35.258381+0000","last_peered":"2023-03-29T14:05:35.258381+0000","last_clean":"2023-03-29T14:05:35.258381+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:35.258381+0000","last_undegraded":"2023-03-29T14:05:35.258381+0000","last_fullsized":"2023-03-29T14:05:35.258381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'111","reported_seq":392,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-03-29T14:05:35.279558+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:35.279558+0000","last_peered":"2023-03-29T14:05:35.279558+0000","last_clean":"2023-03-29T14:05:35.279558+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:35.279558+0000","last_undegraded":"2023-03-29T14:05:35.279558+0000","last_fullsized":"2023-03-29T14:05:35.279558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16471058,"num_objects":44,"num_object_clones":34,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":159,"num_read_kb":92204,"num_write":216,"num_write_kb":64792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":191,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.240360+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:05:35.240360+0000","last_peered":"2023-03-29T14:05:35.240360+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:05:35.240360+0000","last_undegraded":"2023-03-29T14:05:35.240360+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":192,"state":"active+undersized","last_fresh":"2023-03-29T14:05:35.487509+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:05:35.487509+0000","last_peered":"2023-03-29T14:05:35.487509+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:05:35.487509+0000","last_undegraded":"2023-03-29T14:05:35.487509+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"188'101","reported_seq":378,"reported_epoch":191,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:35.225816+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:05:35.225816+0000","last_peered":"2023-03-29T14:05:35.225816+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:05:35.225816+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19866072,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":145,"num_read_kb":83788,"num_write":196,"num_write_kb":57378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":195836880,"num_objects":422,"num_object_clones":330,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":1636,"num_read_kb":971265,"num_write":2144,"num_write_kb":628712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1155,"ondisk_log_size":1155,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":233156,"kb_used_data":220092,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94138684,"statfs":{"total":96636764160,"available":96398012416,"internally_reserved":0,"allocated":225374208,"data_stored":267957961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":266772,"kb_used_data":252748,"kb_used_omap":0,"kb_used_meta":14016,"kb_avail":94105068,"statfs":{"total":96636764160,"available":96363589632,"internally_reserved":0,"allocated":258813952,"data_stored":313736046,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14352384},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":247812,"kb_used_data":234108,"kb_used_omap":0,"kb_used_meta":13696,"kb_avail":94124028,"statfs":{"total":96636764160,"available":96383004672,"internally_reserved":0,"allocated":239726592,"data_stored":291190233,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14024704},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:40.743+0000 7f06504a7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:40.747+0000 7f06504a7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:40.822 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:40.825 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:40.825 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:41.366 INFO:tasks.rados.rados.0.smithi017.stdout:739: done (1 left) 2023-03-29T14:05:41.366 INFO:tasks.rados.rados.0.smithi017.stdout:740: done (0 left) 2023-03-29T14:05:41.366 INFO:tasks.rados.rados.0.smithi017.stdout:741: read oid 17 snap -1 2023-03-29T14:05:41.366 INFO:tasks.rados.rados.0.smithi017.stdout:741: expect (ObjNum 347 snap 82 seq_num 347) 2023-03-29T14:05:41.366 INFO:tasks.rados.rados.0.smithi017.stdout:742: snap_create 2023-03-29T14:05:41.412 INFO:tasks.rados.rados.0.smithi017.stdout:741: expect (ObjNum 347 snap 82 seq_num 347) 2023-03-29T14:05:42.370 INFO:tasks.rados.rados.0.smithi017.stdout:741: done (1 left) 2023-03-29T14:05:42.370 INFO:tasks.rados.rados.0.smithi017.stdout:742: done (0 left) 2023-03-29T14:05:42.370 INFO:tasks.rados.rados.0.smithi017.stdout:743: read oid 29 snap -1 2023-03-29T14:05:42.370 INFO:tasks.rados.rados.0.smithi017.stdout:743: expect (ObjNum 293 snap 78 seq_num 293) 2023-03-29T14:05:42.370 INFO:tasks.rados.rados.0.smithi017.stdout:744: write oid 35 current snap is 89 2023-03-29T14:05:42.370 INFO:tasks.rados.rados.0.smithi017.stdout:744: seq_num 358 ranges {416906=407733,1366083=511456} 2023-03-29T14:05:42.374 INFO:tasks.rados.rados.0.smithi017.stdout:744: writing smithi017107884-35 from 416906 to 824639 tid 1 2023-03-29T14:05:42.380 INFO:tasks.rados.rados.0.smithi017.stdout:744: writing smithi017107884-35 from 1366083 to 1877539 tid 2 2023-03-29T14:05:42.380 INFO:tasks.rados.rados.0.smithi017.stdout:745: read oid 46 snap -1 2023-03-29T14:05:42.380 INFO:tasks.rados.rados.0.smithi017.stdout:745: expect (ObjNum 285 snap 75 seq_num 285) 2023-03-29T14:05:42.381 INFO:tasks.rados.rados.0.smithi017.stdout:746: read oid 9 snap -1 2023-03-29T14:05:42.381 INFO:tasks.rados.rados.0.smithi017.stdout:746: expect (ObjNum 352 snap 86 seq_num 352) 2023-03-29T14:05:42.381 INFO:tasks.rados.rados.0.smithi017.stdout:747: delete oid 30 current snap is 89 2023-03-29T14:05:42.411 INFO:tasks.rados.rados.0.smithi017.stdout:746: expect (ObjNum 352 snap 86 seq_num 352) 2023-03-29T14:05:42.751 INFO:tasks.rados.rados.0.smithi017.stdout:743: expect (ObjNum 293 snap 78 seq_num 293) 2023-03-29T14:05:42.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:42.900 INFO:tasks.rados.rados.0.smithi017.stdout:744: finishing write tid 1 to smithi017107884-35 2023-03-29T14:05:42.900 INFO:tasks.rados.rados.0.smithi017.stdout:744: finishing write tid 2 to smithi017107884-35 2023-03-29T14:05:42.900 INFO:tasks.rados.rados.0.smithi017.stdout:744: finishing write tid 3 to smithi017107884-35 2023-03-29T14:05:42.901 INFO:tasks.rados.rados.0.smithi017.stdout:744: finishing write tid 4 to smithi017107884-35 2023-03-29T14:05:42.901 INFO:tasks.rados.rados.0.smithi017.stdout:744: oid 35 updating version 0 to 56 2023-03-29T14:05:42.901 INFO:tasks.rados.rados.0.smithi017.stdout:744: oid 35 version 56 is already newer than 56 2023-03-29T14:05:42.901 INFO:tasks.rados.rados.0.smithi017.stdout:744: oid 35 version 56 is already newer than 56 2023-03-29T14:05:42.901 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 56 (ObjNum 358 snap 89 seq_num 358) dirty exists 2023-03-29T14:05:42.901 INFO:tasks.rados.rados.0.smithi017.stdout:744: left oid 35 (ObjNum 358 snap 89 seq_num 358) 2023-03-29T14:05:42.901 INFO:tasks.rados.rados.0.smithi017.stdout:745: expect (ObjNum 285 snap 75 seq_num 285) 2023-03-29T14:05:42.957 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:42.955+0000 7f454c75c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:42.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:42.960+0000 7f454c75c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:43.022 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:43.026 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:43.026 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:43.052 INFO:tasks.rados.rados.0.smithi017.stdout:743: done (4 left) 2023-03-29T14:05:43.052 INFO:tasks.rados.rados.0.smithi017.stdout:744: done (3 left) 2023-03-29T14:05:43.053 INFO:tasks.rados.rados.0.smithi017.stdout:745: done (2 left) 2023-03-29T14:05:43.053 INFO:tasks.rados.rados.0.smithi017.stdout:746: done (1 left) 2023-03-29T14:05:43.053 INFO:tasks.rados.rados.0.smithi017.stdout:747: done (0 left) 2023-03-29T14:05:43.053 INFO:tasks.rados.rados.0.smithi017.stdout:748: write oid 15 current snap is 89 2023-03-29T14:05:43.053 INFO:tasks.rados.rados.0.smithi017.stdout:748: seq_num 360 ranges {450520=558231,1483047=515125} 2023-03-29T14:05:43.058 INFO:tasks.rados.rados.0.smithi017.stdout:748: writing smithi017107884-15 from 450520 to 1008751 tid 1 2023-03-29T14:05:43.063 INFO:tasks.rados.rados.0.smithi017.stdout:748: writing smithi017107884-15 from 1483047 to 1998172 tid 2 2023-03-29T14:05:43.063 INFO:tasks.rados.rados.0.smithi017.stdout:749: read oid 5 snap 88 2023-03-29T14:05:43.064 INFO:tasks.rados.rados.0.smithi017.stdout:749: expect deleted 2023-03-29T14:05:43.064 INFO:tasks.rados.rados.0.smithi017.stdout:750: write oid 25 current snap is 89 2023-03-29T14:05:43.064 INFO:tasks.rados.rados.0.smithi017.stdout:750: seq_num 361 ranges {467327=633480,1825176=242151} 2023-03-29T14:05:43.070 INFO:tasks.rados.rados.0.smithi017.stdout:750: writing smithi017107884-25 from 467327 to 1100807 tid 1 2023-03-29T14:05:43.075 INFO:tasks.rados.rados.0.smithi017.stdout:750: writing smithi017107884-25 from 1825176 to 2067327 tid 2 2023-03-29T14:05:43.075 INFO:tasks.rados.rados.0.smithi017.stdout:751: write (excl) oid 44 current snap is 89 2023-03-29T14:05:43.075 INFO:tasks.rados.rados.0.smithi017.stdout:751: seq_num 362 ranges {484134=708729,1883658=200476} 2023-03-29T14:05:43.081 INFO:tasks.rados.rados.0.smithi017.stdout:751: writing smithi017107884-44 from 484134 to 1192863 tid 1 2023-03-29T14:05:43.087 INFO:tasks.rados.rados.0.smithi017.stdout:751: writing smithi017107884-44 from 1883658 to 2084134 tid 2 2023-03-29T14:05:43.087 INFO:tasks.rados.rados.0.smithi017.stdout:752: delete oid 22 current snap is 89 2023-03-29T14:05:43.090 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~33s 2023-03-29T14:05:43.102 INFO:tasks.rados.rados.0.smithi017.stdout:748: finishing write tid 1 to smithi017107884-15 2023-03-29T14:05:43.108 INFO:tasks.rados.rados.0.smithi017.stdout:750: finishing write tid 1 to smithi017107884-25 2023-03-29T14:05:43.137 INFO:tasks.rados.rados.0.smithi017.stdout:748: finishing write tid 2 to smithi017107884-15 2023-03-29T14:05:43.159 INFO:tasks.rados.rados.0.smithi017.stdout:748: finishing write tid 3 to smithi017107884-15 2023-03-29T14:05:43.159 INFO:tasks.rados.rados.0.smithi017.stdout:750: finishing write tid 2 to smithi017107884-25 2023-03-29T14:05:43.159 INFO:tasks.rados.rados.0.smithi017.stdout:750: finishing write tid 3 to smithi017107884-25 2023-03-29T14:05:43.160 INFO:tasks.rados.rados.0.smithi017.stdout:750: finishing write tid 4 to smithi017107884-25 2023-03-29T14:05:43.161 INFO:tasks.rados.rados.0.smithi017.stdout:750: oid 25 updating version 0 to 111 2023-03-29T14:05:43.161 INFO:tasks.rados.rados.0.smithi017.stdout:750: oid 25 version 111 is already newer than 111 2023-03-29T14:05:43.161 INFO:tasks.rados.rados.0.smithi017.stdout:750: oid 25 version 111 is already newer than 111 2023-03-29T14:05:43.161 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 111 (ObjNum 361 snap 89 seq_num 361) dirty exists 2023-03-29T14:05:43.161 INFO:tasks.rados.rados.0.smithi017.stdout:750: left oid 25 (ObjNum 361 snap 89 seq_num 361) 2023-03-29T14:05:43.163 INFO:tasks.rados.rados.0.smithi017.stdout:751: finishing write tid 1 to smithi017107884-44 2023-03-29T14:05:43.164 INFO:tasks.rados.rados.0.smithi017.stdout:749: done (4 left) 2023-03-29T14:05:43.164 INFO:tasks.rados.rados.0.smithi017.stdout:750: done (3 left) 2023-03-29T14:05:43.164 INFO:tasks.rados.rados.0.smithi017.stdout:752: done (2 left) 2023-03-29T14:05:43.164 INFO:tasks.rados.rados.0.smithi017.stdout:753: delete oid 45 current snap is 89 2023-03-29T14:05:43.166 INFO:tasks.rados.rados.0.smithi017.stdout:748: finishing write tid 4 to smithi017107884-15 2023-03-29T14:05:43.166 INFO:tasks.rados.rados.0.smithi017.stdout:748: oid 15 updating version 0 to 162 2023-03-29T14:05:43.166 INFO:tasks.rados.rados.0.smithi017.stdout:748: oid 15 version 162 is already newer than 162 2023-03-29T14:05:43.166 INFO:tasks.rados.rados.0.smithi017.stdout:748: oid 15 version 162 is already newer than 162 2023-03-29T14:05:43.166 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 162 (ObjNum 360 snap 89 seq_num 360) dirty exists 2023-03-29T14:05:43.166 INFO:tasks.rados.rados.0.smithi017.stdout:748: left oid 15 (ObjNum 360 snap 89 seq_num 360) 2023-03-29T14:05:43.189 INFO:tasks.rados.rados.0.smithi017.stdout:751: finishing write tid 2 to smithi017107884-44 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:751: finishing write tid 3 to smithi017107884-44 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:751: finishing write tid 4 to smithi017107884-44 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:751: oid 44 updating version 0 to 57 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:751: oid 44 updating version 57 to 60 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:751: oid 44 version 60 is already newer than 60 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 60 (ObjNum 362 snap 89 seq_num 362) dirty exists 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:751: left oid 44 (ObjNum 362 snap 89 seq_num 362) 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:748: done (2 left) 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:751: done (1 left) 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:753: done (0 left) 2023-03-29T14:05:43.230 INFO:tasks.rados.rados.0.smithi017.stdout:754: snap_create 2023-03-29T14:05:44.213 INFO:tasks.rados.rados.0.smithi017.stdout:754: done (0 left) 2023-03-29T14:05:44.213 INFO:tasks.rados.rados.0.smithi017.stdout:755: snap_remove snap 88 2023-03-29T14:05:45.027 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:45.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:45.156+0000 7fdfa82f7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:45.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:45.160+0000 7fdfa82f7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:45.215 INFO:tasks.rados.rados.0.smithi017.stdout:755: done (0 left) 2023-03-29T14:05:45.215 INFO:tasks.rados.rados.0.smithi017.stdout:756: snap_remove snap 89 2023-03-29T14:05:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:45.229 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:45.230 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:45.735 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:45.736 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:45.736 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:45.862+0000 7f1767439700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:45.869 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:45.869+0000 7f1767439700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:46.072 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:46.074 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:46.085 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-03-29T14:05:45.213501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190967096,"num_objects":434,"num_object_clones":338,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":34,"num_read":1800,"num_read_kb":991274,"num_write":2446,"num_write_kb":641500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1272,"ondisk_log_size":1272,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":919520,"kb_used_data":866304,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":376567840,"statfs":{"total":386547056640,"available":385605468160,"internally_reserved":0,"allocated":887095296,"data_stored":1067840929,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6050520,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":22,"num_read_kb":19889,"num_write":28,"num_write_kb":9000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.851877"},"pg_stats":[{"pgid":"3.f","version":"173'40","reported_seq":20,"reported_epoch":197,"state":"active+undersized","last_fresh":"2023-03-29T14:05:40.375479+0000","last_change":"2023-03-29T14:05:30.457299+0000","last_active":"2023-03-29T14:05:40.375479+0000","last_peered":"2023-03-29T14:05:40.375479+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:05:40.375479+0000","last_undegraded":"2023-03-29T14:05:40.375479+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":195,"reported_epoch":196,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:39.415556+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:05:39.415556+0000","last_peered":"2023-03-29T14:05:39.415556+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:05:39.415556+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"188'104","reported_seq":391,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-03-29T14:05:41.405427+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:41.405427+0000","last_peered":"2023-03-29T14:05:41.405427+0000","last_clean":"2023-03-29T14:05:41.405427+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:41.405427+0000","last_undegraded":"2023-03-29T14:05:41.405427+0000","last_fullsized":"2023-03-29T14:05:41.405427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13949942,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":149,"num_read_kb":109409,"num_write":202,"num_write_kb":60172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"192'108","reported_seq":401,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-03-29T14:05:40.376929+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:40.376929+0000","last_peered":"2023-03-29T14:05:40.376929+0000","last_clean":"2023-03-29T14:05:40.376929+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:40.376929+0000","last_undegraded":"2023-03-29T14:05:40.376929+0000","last_fullsized":"2023-03-29T14:05:40.376929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15260888,"num_objects":48,"num_object_clones":40,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":169,"num_read_kb":104933,"num_write":208,"num_write_kb":59016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"188'99","reported_seq":373,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-03-29T14:05:39.439508+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:39.439508+0000","last_peered":"2023-03-29T14:05:39.439508+0000","last_clean":"2023-03-29T14:05:39.439508+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:39.439508+0000","last_undegraded":"2023-03-29T14:05:39.439508+0000","last_fullsized":"2023-03-29T14:05:39.439508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185570,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":71229,"num_write":204,"num_write_kb":61526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":158,"reported_epoch":197,"state":"active+undersized","last_fresh":"2023-03-29T14:05:40.385586+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:05:40.385586+0000","last_peered":"2023-03-29T14:05:40.385586+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:05:40.385586+0000","last_undegraded":"2023-03-29T14:05:40.385586+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"191'78","reported_seq":330,"reported_epoch":197,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:40.390257+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:05:40.390257+0000","last_peered":"2023-03-29T14:05:40.390257+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:05:40.390257+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15361730,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":107,"num_read_kb":51656,"num_write":146,"num_write_kb":43166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":32}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":275,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-03-29T14:05:40.383030+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:40.383030+0000","last_peered":"2023-03-29T14:05:40.383030+0000","last_clean":"2023-03-29T14:05:40.383030+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:40.383030+0000","last_undegraded":"2023-03-29T14:05:40.383030+0000","last_fullsized":"2023-03-29T14:05:40.383030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":157,"reported_epoch":196,"state":"active+undersized","last_fresh":"2023-03-29T14:05:39.422446+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:05:39.422446+0000","last_peered":"2023-03-29T14:05:39.422446+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:05:39.422446+0000","last_undegraded":"2023-03-29T14:05:39.422446+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":19,"reported_epoch":196,"state":"active+undersized","last_fresh":"2023-03-29T14:05:38.282248+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:05:38.282248+0000","last_peered":"2023-03-29T14:05:38.282248+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:05:38.282248+0000","last_undegraded":"2023-03-29T14:05:38.282248+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":161,"reported_epoch":197,"state":"active+undersized","last_fresh":"2023-03-29T14:05:40.382120+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:05:40.382120+0000","last_peered":"2023-03-29T14:05:40.382120+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:05:40.382120+0000","last_undegraded":"2023-03-29T14:05:40.382120+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":281,"reported_epoch":196,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:38.317353+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:05:38.317353+0000","last_peered":"2023-03-29T14:05:38.317353+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:05:38.317353+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":40088,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":196,"state":"active+undersized","last_fresh":"2023-03-29T14:05:39.436409+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:05:39.436409+0000","last_peered":"2023-03-29T14:05:39.436409+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:05:39.436409+0000","last_undegraded":"2023-03-29T14:05:39.436409+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"196'81","reported_seq":334,"reported_epoch":196,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:40.512553+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:05:40.512553+0000","last_peered":"2023-03-29T14:05:40.512553+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:05:40.512553+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193792,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":45592,"num_write":170,"num_write_kb":51308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":161,"reported_epoch":197,"state":"active+undersized","last_fresh":"2023-03-29T14:05:40.388631+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:05:40.388631+0000","last_peered":"2023-03-29T14:05:40.388631+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:05:40.388631+0000","last_undegraded":"2023-03-29T14:05:40.388631+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":382,"reported_epoch":196,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:39.434529+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:05:39.434529+0000","last_peered":"2023-03-29T14:05:39.434529+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:05:39.434529+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"175'68","reported_seq":340,"reported_epoch":197,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:40.374717+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:05:40.374717+0000","last_peered":"2023-03-29T14:05:40.374717+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:05:40.374717+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260690,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":160,"num_read_kb":103515,"num_write":134,"num_write_kb":38030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":197,"state":"active+undersized","last_fresh":"2023-03-29T14:05:40.377470+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:05:40.377470+0000","last_peered":"2023-03-29T14:05:40.377470+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:05:40.377470+0000","last_undegraded":"2023-03-29T14:05:40.377470+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"191'158","reported_seq":529,"reported_epoch":196,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:38.331370+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:05:38.331370+0000","last_peered":"2023-03-29T14:05:38.331370+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:05:38.331370+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18689582,"num_objects":60,"num_object_clones":46,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":269,"num_read_kb":174545,"num_write":302,"num_write_kb":87244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":196,"state":"active+undersized","last_fresh":"2023-03-29T14:05:39.410530+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:05:39.410530+0000","last_peered":"2023-03-29T14:05:39.410530+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:05:39.410530+0000","last_undegraded":"2023-03-29T14:05:39.410530+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"166'52","reported_seq":272,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-03-29T14:05:39.443707+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:39.443707+0000","last_peered":"2023-03-29T14:05:39.443707+0000","last_clean":"2023-03-29T14:05:39.443707+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:39.443707+0000","last_undegraded":"2023-03-29T14:05:39.443707+0000","last_fullsized":"2023-03-29T14:05:39.443707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10857454,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":47063,"num_write":110,"num_write_kb":32110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"195'118","reported_seq":408,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-03-29T14:05:39.510480+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:39.510480+0000","last_peered":"2023-03-29T14:05:39.510480+0000","last_clean":"2023-03-29T14:05:39.510480+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:39.510480+0000","last_undegraded":"2023-03-29T14:05:39.510480+0000","last_fullsized":"2023-03-29T14:05:39.510480+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12773452,"num_objects":48,"num_object_clones":38,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":92205,"num_write":226,"num_write_kb":68272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":196,"state":"active+undersized","last_fresh":"2023-03-29T14:05:39.427576+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:05:39.427576+0000","last_peered":"2023-03-29T14:05:39.427576+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:05:39.427576+0000","last_undegraded":"2023-03-29T14:05:39.427576+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":161,"reported_epoch":197,"state":"active+undersized","last_fresh":"2023-03-29T14:05:40.384841+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:05:40.384841+0000","last_peered":"2023-03-29T14:05:40.384841+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:05:40.384841+0000","last_undegraded":"2023-03-29T14:05:40.384841+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"192'106","reported_seq":391,"reported_epoch":196,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:38.272516+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:05:38.272516+0000","last_peered":"2023-03-29T14:05:38.272516+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:05:38.272516+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21008948,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":148,"num_read_kb":83789,"num_write":206,"num_write_kb":60894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":189786360,"num_objects":430,"num_object_clones":338,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":34,"num_read":1658,"num_read_kb":991154,"num_write":2172,"num_write_kb":637712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1175,"ondisk_log_size":1175,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":236636,"kb_used_data":223508,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94135204,"statfs":{"total":96636764160,"available":96394448896,"internally_reserved":0,"allocated":228872192,"data_stored":272777896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":272072,"kb_used_data":257920,"kb_used_omap":0,"kb_used_meta":14144,"kb_avail":94099768,"statfs":{"total":96636764160,"available":96358162432,"internally_reserved":0,"allocated":264110080,"data_stored":321140984,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14483456},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":254868,"kb_used_data":241036,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94116972,"statfs":{"total":96636764160,"available":96375779328,"internally_reserved":0,"allocated":246820864,"data_stored":300689824,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:46.220 INFO:tasks.rados.rados.0.smithi017.stdout:756: done (0 left) 2023-03-29T14:05:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:757: write oid 26 current snap is 90 2023-03-29T14:05:46.221 INFO:tasks.rados.rados.0.smithi017.stdout:757: seq_num 365 ranges {534555=650829,1659104=475451} 2023-03-29T14:05:46.227 INFO:tasks.rados.rados.0.smithi017.stdout:757: writing smithi017107884-26 from 534555 to 1185384 tid 1 2023-03-29T14:05:46.233 INFO:tasks.rados.rados.0.smithi017.stdout:757: writing smithi017107884-26 from 1659104 to 2134555 tid 2 2023-03-29T14:05:46.233 INFO:tasks.rados.rados.0.smithi017.stdout:758: read oid 29 snap -1 2023-03-29T14:05:46.233 INFO:tasks.rados.rados.0.smithi017.stdout:758: expect (ObjNum 293 snap 78 seq_num 293) 2023-03-29T14:05:46.233 INFO:tasks.rados.rados.0.smithi017.stdout:759: write (excl) oid 9 current snap is 90 2023-03-29T14:05:46.233 INFO:tasks.rados.rados.0.smithi017.stdout:759: seq_num 366 ranges {551362=726078,1717586=409779} 2023-03-29T14:05:46.239 INFO:tasks.rados.rados.0.smithi017.stdout:759: writing smithi017107884-9 from 551362 to 1277440 tid 1 2023-03-29T14:05:46.244 INFO:tasks.rados.rados.0.smithi017.stdout:759: writing smithi017107884-9 from 1717586 to 2127365 tid 2 2023-03-29T14:05:46.244 INFO:tasks.rados.rados.0.smithi017.stdout:760: snap_remove snap 87 2023-03-29T14:05:47.225 INFO:tasks.rados.rados.0.smithi017.stdout:760: done (3 left) 2023-03-29T14:05:47.225 INFO:tasks.rados.rados.0.smithi017.stdout:761: delete oid 5 current snap is 90 2023-03-29T14:05:47.226 INFO:tasks.rados.rados.0.smithi017.stdout:757: finishing write tid 1 to smithi017107884-26 2023-03-29T14:05:47.226 INFO:tasks.rados.rados.0.smithi017.stdout:757: finishing write tid 2 to smithi017107884-26 2023-03-29T14:05:47.226 INFO:tasks.rados.rados.0.smithi017.stdout:757: finishing write tid 3 to smithi017107884-26 2023-03-29T14:05:47.226 INFO:tasks.rados.rados.0.smithi017.stdout:757: finishing write tid 4 to smithi017107884-26 2023-03-29T14:05:47.226 INFO:tasks.rados.rados.0.smithi017.stdout:757: oid 26 updating version 0 to 71 2023-03-29T14:05:47.227 INFO:tasks.rados.rados.0.smithi017.stdout:757: oid 26 version 71 is already newer than 71 2023-03-29T14:05:47.227 INFO:tasks.rados.rados.0.smithi017.stdout:757: oid 26 version 71 is already newer than 71 2023-03-29T14:05:47.227 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 71 (ObjNum 365 snap 90 seq_num 365) dirty exists 2023-03-29T14:05:47.227 INFO:tasks.rados.rados.0.smithi017.stdout:757: left oid 26 (ObjNum 365 snap 90 seq_num 365) 2023-03-29T14:05:47.227 INFO:tasks.rados.rados.0.smithi017.stdout:758: expect (ObjNum 293 snap 78 seq_num 293) 2023-03-29T14:05:47.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:47.359+0000 7f5492eaa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:47.363+0000 7f5492eaa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:47.452 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:47.455 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:47.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:47.456 INFO:tasks.rados.rados.0.smithi017.stdout:759: finishing write tid 1 to smithi017107884-9 2023-03-29T14:05:47.456 INFO:tasks.rados.rados.0.smithi017.stdout:759: finishing write tid 2 to smithi017107884-9 2023-03-29T14:05:47.456 INFO:tasks.rados.rados.0.smithi017.stdout:759: finishing write tid 3 to smithi017107884-9 2023-03-29T14:05:47.456 INFO:tasks.rados.rados.0.smithi017.stdout:759: finishing write tid 4 to smithi017107884-9 2023-03-29T14:05:47.456 INFO:tasks.rados.rados.0.smithi017.stdout:759: oid 9 updating version 0 to 115 2023-03-29T14:05:47.457 INFO:tasks.rados.rados.0.smithi017.stdout:759: oid 9 version 115 is already newer than 112 2023-03-29T14:05:47.457 INFO:tasks.rados.rados.0.smithi017.stdout:759: oid 9 version 115 is already newer than 115 2023-03-29T14:05:47.457 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 115 (ObjNum 366 snap 90 seq_num 366) dirty exists 2023-03-29T14:05:47.457 INFO:tasks.rados.rados.0.smithi017.stdout:759: left oid 9 (ObjNum 366 snap 90 seq_num 366) 2023-03-29T14:05:47.457 INFO:tasks.rados.rados.0.smithi017.stdout:757: done (3 left) 2023-03-29T14:05:47.457 INFO:tasks.rados.rados.0.smithi017.stdout:758: done (2 left) 2023-03-29T14:05:47.458 INFO:tasks.rados.rados.0.smithi017.stdout:759: done (1 left) 2023-03-29T14:05:47.458 INFO:tasks.rados.rados.0.smithi017.stdout:761: done (0 left) 2023-03-29T14:05:47.458 INFO:tasks.rados.rados.0.smithi017.stdout:762: write (excl) oid 16 current snap is 90 2023-03-29T14:05:47.458 INFO:tasks.rados.rados.0.smithi017.stdout:762: seq_num 368 ranges {584976=476576,1834550=350426} 2023-03-29T14:05:47.461 INFO:tasks.rados.rados.0.smithi017.stdout:762: writing smithi017107884-16 from 584976 to 1061552 tid 1 2023-03-29T14:05:47.468 INFO:tasks.rados.rados.0.smithi017.stdout:762: writing smithi017107884-16 from 1834550 to 2184976 tid 2 2023-03-29T14:05:47.468 INFO:tasks.rados.rados.0.smithi017.stdout:763: write oid 31 current snap is 90 2023-03-29T14:05:47.468 INFO:tasks.rados.rados.0.smithi017.stdout:763: seq_num 369 ranges {601783=551825,1776679=425104} 2023-03-29T14:05:47.474 INFO:tasks.rados.rados.0.smithi017.stdout:763: writing smithi017107884-31 from 601783 to 1153608 tid 1 2023-03-29T14:05:47.480 INFO:tasks.rados.rados.0.smithi017.stdout:763: writing smithi017107884-31 from 1776679 to 2201783 tid 2 2023-03-29T14:05:47.480 INFO:tasks.rados.rados.0.smithi017.stdout:764: write (excl) oid 30 current snap is 90 2023-03-29T14:05:47.480 INFO:tasks.rados.rados.0.smithi017.stdout:764: seq_num 370 ranges {618590=627074,1835161=383429} 2023-03-29T14:05:47.487 INFO:tasks.rados.rados.0.smithi017.stdout:764: writing smithi017107884-30 from 618590 to 1245664 tid 1 2023-03-29T14:05:47.492 INFO:tasks.rados.rados.0.smithi017.stdout:764: writing smithi017107884-30 from 1835161 to 2218590 tid 2 2023-03-29T14:05:47.492 INFO:tasks.rados.rados.0.smithi017.stdout:765: write (excl) oid 35 current snap is 90 2023-03-29T14:05:47.493 INFO:tasks.rados.rados.0.smithi017.stdout:765: seq_num 371 ranges {635397=702323,1893643=341754} 2023-03-29T14:05:47.499 INFO:tasks.rados.rados.0.smithi017.stdout:765: writing smithi017107884-35 from 635397 to 1337720 tid 1 2023-03-29T14:05:47.505 INFO:tasks.rados.rados.0.smithi017.stdout:765: writing smithi017107884-35 from 1893643 to 2235397 tid 2 2023-03-29T14:05:47.505 INFO:tasks.rados.rados.0.smithi017.stdout:766: write (excl) oid 10 current snap is 90 2023-03-29T14:05:47.505 INFO:tasks.rados.rados.0.smithi017.stdout:766: seq_num 372 ranges {652204=777572,1952125=300079} 2023-03-29T14:05:47.512 INFO:tasks.rados.rados.0.smithi017.stdout:766: writing smithi017107884-10 from 652204 to 1429776 tid 1 2023-03-29T14:05:47.517 INFO:tasks.rados.rados.0.smithi017.stdout:766: writing smithi017107884-10 from 1952125 to 2252204 tid 2 2023-03-29T14:05:47.517 INFO:tasks.rados.rados.0.smithi017.stdout:767: write (excl) oid 15 current snap is 90 2023-03-29T14:05:47.517 INFO:tasks.rados.rados.0.smithi017.stdout:767: seq_num 373 ranges {669011=569174,2010607=258404} 2023-03-29T14:05:47.524 INFO:tasks.rados.rados.0.smithi017.stdout:767: writing smithi017107884-15 from 669011 to 1238185 tid 1 2023-03-29T14:05:47.530 INFO:tasks.rados.rados.0.smithi017.stdout:767: writing smithi017107884-15 from 2010607 to 2269011 tid 2 2023-03-29T14:05:47.530 INFO:tasks.rados.rados.0.smithi017.stdout:768: write (excl) oid 7 current snap is 90 2023-03-29T14:05:47.530 INFO:tasks.rados.rados.0.smithi017.stdout:768: seq_num 374 ranges {685818=644423,2069089=216729} 2023-03-29T14:05:47.537 INFO:tasks.rados.rados.0.smithi017.stdout:768: writing smithi017107884-7 from 685818 to 1330241 tid 1 2023-03-29T14:05:47.543 INFO:tasks.rados.rados.0.smithi017.stdout:768: writing smithi017107884-7 from 2069089 to 2285818 tid 2 2023-03-29T14:05:47.543 INFO:tasks.rados.rados.0.smithi017.stdout:769: read oid 49 snap 83 2023-03-29T14:05:47.543 INFO:tasks.rados.rados.0.smithi017.stdout:769: expect deleted 2023-03-29T14:05:47.543 INFO:tasks.rados.rados.0.smithi017.stdout:770: snap_remove snap 86 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:770: done (8 left) 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:771: read oid 47 snap 83 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:771: expect (ObjNum 263 snap 70 seq_num 263) 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:772: read oid 44 snap -1 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:772: expect (ObjNum 362 snap 89 seq_num 362) 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:773: write (excl) oid 27 current snap is 90 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:763: finishing write tid 1 to smithi017107884-31 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:763: finishing write tid 2 to smithi017107884-31 2023-03-29T14:05:48.229 INFO:tasks.rados.rados.0.smithi017.stdout:763: finishing write tid 3 to smithi017107884-31 2023-03-29T14:05:48.230 INFO:tasks.rados.rados.0.smithi017.stdout:773: seq_num 375 ranges {702625=719672,2127571=175054} 2023-03-29T14:05:48.239 INFO:tasks.rados.rados.0.smithi017.stdout:773: writing smithi017107884-27 from 702625 to 1422297 tid 1 2023-03-29T14:05:48.244 INFO:tasks.rados.rados.0.smithi017.stdout:773: writing smithi017107884-27 from 2127571 to 2302625 tid 2 2023-03-29T14:05:48.245 INFO:tasks.rados.rados.0.smithi017.stdout:774: write oid 13 current snap is 90 2023-03-29T14:05:48.245 INFO:tasks.rados.rados.0.smithi017.stdout:774: seq_num 376 ranges {719432=794921,2186053=133379} 2023-03-29T14:05:48.256 INFO:tasks.rados.rados.0.smithi017.stdout:774: writing smithi017107884-13 from 719432 to 1514353 tid 1 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:774: writing smithi017107884-13 from 2186053 to 2319432 tid 2 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:775: snap_create 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:763: finishing write tid 4 to smithi017107884-31 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:763: oid 31 updating version 0 to 82 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:763: oid 31 version 82 is already newer than 82 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:763: oid 31 version 82 is already newer than 82 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 82 (ObjNum 369 snap 90 seq_num 369) dirty exists 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:763: left oid 31 (ObjNum 369 snap 90 seq_num 369) 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:762: finishing write tid 1 to smithi017107884-16 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:766: finishing write tid 1 to smithi017107884-10 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:765: finishing write tid 1 to smithi017107884-35 2023-03-29T14:05:48.262 INFO:tasks.rados.rados.0.smithi017.stdout:768: finishing write tid 1 to smithi017107884-7 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:764: finishing write tid 1 to smithi017107884-30 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:762: finishing write tid 2 to smithi017107884-16 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:762: finishing write tid 3 to smithi017107884-16 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:762: finishing write tid 4 to smithi017107884-16 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:762: oid 16 updating version 0 to 43 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:762: oid 16 updating version 43 to 45 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:762: oid 16 version 45 is already newer than 45 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 45 (ObjNum 368 snap 90 seq_num 368) dirty exists 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:762: left oid 16 (ObjNum 368 snap 90 seq_num 368) 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:767: finishing write tid 1 to smithi017107884-15 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:765: finishing write tid 2 to smithi017107884-35 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:765: finishing write tid 3 to smithi017107884-35 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:765: finishing write tid 4 to smithi017107884-35 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:765: oid 35 updating version 0 to 61 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:765: oid 35 updating version 61 to 63 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:765: oid 35 version 63 is already newer than 63 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 63 (ObjNum 371 snap 90 seq_num 371) dirty exists 2023-03-29T14:05:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:765: left oid 35 (ObjNum 371 snap 90 seq_num 371) 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:766: finishing write tid 2 to smithi017107884-10 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:766: finishing write tid 3 to smithi017107884-10 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:766: finishing write tid 4 to smithi017107884-10 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:766: oid 10 updating version 0 to 105 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:766: oid 10 updating version 105 to 108 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:766: oid 10 version 108 is already newer than 108 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 108 (ObjNum 372 snap 90 seq_num 372) dirty exists 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:766: left oid 10 (ObjNum 372 snap 90 seq_num 372) 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:764: finishing write tid 2 to smithi017107884-30 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:764: finishing write tid 3 to smithi017107884-30 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:764: finishing write tid 4 to smithi017107884-30 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:764: oid 30 updating version 0 to 42 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:764: oid 30 version 42 is already newer than 40 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:764: oid 30 version 42 is already newer than 42 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 42 (ObjNum 370 snap 90 seq_num 370) dirty exists 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:764: left oid 30 (ObjNum 370 snap 90 seq_num 370) 2023-03-29T14:05:48.264 INFO:tasks.rados.rados.0.smithi017.stdout:767: finishing write tid 2 to smithi017107884-15 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:767: finishing write tid 3 to smithi017107884-15 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:767: finishing write tid 4 to smithi017107884-15 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:767: oid 15 updating version 0 to 165 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:767: oid 15 version 165 is already newer than 163 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:767: oid 15 version 165 is already newer than 165 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 165 (ObjNum 373 snap 90 seq_num 373) dirty exists 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:767: left oid 15 (ObjNum 373 snap 90 seq_num 373) 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:768: finishing write tid 2 to smithi017107884-7 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:768: finishing write tid 3 to smithi017107884-7 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:768: finishing write tid 4 to smithi017107884-7 2023-03-29T14:05:48.265 INFO:tasks.rados.rados.0.smithi017.stdout:768: oid 7 updating version 0 to 55 2023-03-29T14:05:48.266 INFO:tasks.rados.rados.0.smithi017.stdout:768: oid 7 version 55 is already newer than 55 2023-03-29T14:05:48.266 INFO:tasks.rados.rados.0.smithi017.stdout:768: oid 7 version 55 is already newer than 53 2023-03-29T14:05:48.266 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 55 (ObjNum 374 snap 90 seq_num 374) dirty exists 2023-03-29T14:05:48.266 INFO:tasks.rados.rados.0.smithi017.stdout:768: left oid 7 (ObjNum 374 snap 90 seq_num 374) 2023-03-29T14:05:48.349 INFO:tasks.rados.rados.0.smithi017.stdout:773: finishing write tid 1 to smithi017107884-27 2023-03-29T14:05:48.393 INFO:tasks.rados.rados.0.smithi017.stdout:772: expect (ObjNum 362 snap 89 seq_num 362) 2023-03-29T14:05:48.503 INFO:tasks.rados.rados.0.smithi017.stdout:773: finishing write tid 2 to smithi017107884-27 2023-03-29T14:05:48.503 INFO:tasks.rados.rados.0.smithi017.stdout:773: finishing write tid 3 to smithi017107884-27 2023-03-29T14:05:48.503 INFO:tasks.rados.rados.0.smithi017.stdout:771: expect (ObjNum 263 snap 70 seq_num 263) 2023-03-29T14:05:48.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~39s 2023-03-29T14:05:48.597 INFO:tasks.rados.rados.0.smithi017.stdout:773: finishing write tid 4 to smithi017107884-27 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:773: oid 27 updating version 0 to 74 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:773: oid 27 version 74 is already newer than 72 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:773: oid 27 version 74 is already newer than 74 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 74 (ObjNum 375 snap 90 seq_num 375) dirty exists 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:773: left oid 27 (ObjNum 375 snap 90 seq_num 375) 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: finishing write tid 1 to smithi017107884-13 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: finishing write tid 2 to smithi017107884-13 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: finishing write tid 3 to smithi017107884-13 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: finishing write tid 4 to smithi017107884-13 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: oid 13 updating version 0 to 122 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: oid 13 version 122 is already newer than 122 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: oid 13 version 122 is already newer than 122 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 122 (ObjNum 376 snap 90 seq_num 376) dirty exists 2023-03-29T14:05:48.598 INFO:tasks.rados.rados.0.smithi017.stdout:774: left oid 13 (ObjNum 376 snap 90 seq_num 376) 2023-03-29T14:05:49.231 INFO:tasks.rados.rados.0.smithi017.stdout:762: done (12 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:763: done (11 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:764: done (10 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:765: done (9 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:766: done (8 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:767: done (7 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:768: done (6 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:769: done (5 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:771: done (4 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:772: done (3 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:773: done (2 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:774: done (1 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:775: done (0 left) 2023-03-29T14:05:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:776: delete oid 31 current snap is 91 2023-03-29T14:05:49.277 INFO:tasks.rados.rados.0.smithi017.stdout:776: done (0 left) 2023-03-29T14:05:49.277 INFO:tasks.rados.rados.0.smithi017.stdout:777: snap_create 2023-03-29T14:05:49.456 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:49.583+0000 7f809ac06700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:49.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:49.587+0000 7f809ac06700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:49.649 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:49.652 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:49.652 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:50.235 INFO:tasks.rados.rados.0.smithi017.stdout:777: done (0 left) 2023-03-29T14:05:50.235 INFO:tasks.rados.rados.0.smithi017.stdout:778: write oid 17 current snap is 92 2023-03-29T14:05:50.235 INFO:tasks.rados.rados.0.smithi017.stdout:778: seq_num 378 ranges {753046=545419,1786664=431793} 2023-03-29T14:05:50.242 INFO:tasks.rados.rados.0.smithi017.stdout:778: writing smithi017107884-17 from 753046 to 1298465 tid 1 2023-03-29T14:05:50.247 INFO:tasks.rados.rados.0.smithi017.stdout:778: writing smithi017107884-17 from 1786664 to 2218457 tid 2 2023-03-29T14:05:50.247 INFO:tasks.rados.rados.0.smithi017.stdout:779: write oid 21 current snap is 92 2023-03-29T14:05:50.247 INFO:tasks.rados.rados.0.smithi017.stdout:779: seq_num 379 ranges {769853=620668,1845146=524707} 2023-03-29T14:05:50.254 INFO:tasks.rados.rados.0.smithi017.stdout:779: writing smithi017107884-21 from 769853 to 1390521 tid 1 2023-03-29T14:05:50.260 INFO:tasks.rados.rados.0.smithi017.stdout:779: writing smithi017107884-21 from 1845146 to 2369853 tid 2 2023-03-29T14:05:50.260 INFO:tasks.rados.rados.0.smithi017.stdout:780: write (excl) oid 16 current snap is 92 2023-03-29T14:05:50.260 INFO:tasks.rados.rados.0.smithi017.stdout:780: seq_num 380 ranges {786660=695917,1903628=435462} 2023-03-29T14:05:50.268 INFO:tasks.rados.rados.0.smithi017.stdout:780: writing smithi017107884-16 from 786660 to 1482577 tid 1 2023-03-29T14:05:50.273 INFO:tasks.rados.rados.0.smithi017.stdout:780: writing smithi017107884-16 from 1903628 to 2339090 tid 2 2023-03-29T14:05:50.273 INFO:tasks.rados.rados.0.smithi017.stdout:781: write (excl) oid 25 current snap is 92 2023-03-29T14:05:50.273 INFO:tasks.rados.rados.0.smithi017.stdout:781: seq_num 381 ranges {403467=487519,1562110=579120} 2023-03-29T14:05:50.278 INFO:tasks.rados.rados.0.smithi017.stdout:781: writing smithi017107884-25 from 403467 to 890986 tid 1 2023-03-29T14:05:50.286 INFO:tasks.rados.rados.0.smithi017.stdout:781: writing smithi017107884-25 from 1562110 to 2141230 tid 2 2023-03-29T14:05:50.286 INFO:tasks.rados.rados.0.smithi017.stdout:782: write oid 47 current snap is 92 2023-03-29T14:05:50.286 INFO:tasks.rados.rados.0.smithi017.stdout:782: seq_num 382 ranges {420274=562768,1620592=439131} 2023-03-29T14:05:50.291 INFO:tasks.rados.rados.0.smithi017.stdout:782: writing smithi017107884-47 from 420274 to 983042 tid 1 2023-03-29T14:05:50.298 INFO:tasks.rados.rados.0.smithi017.stdout:782: writing smithi017107884-47 from 1620592 to 2059723 tid 2 2023-03-29T14:05:50.298 INFO:tasks.rados.rados.0.smithi017.stdout:783: delete oid 22 current snap is 92 2023-03-29T14:05:50.312 INFO:tasks.rados.rados.0.smithi017.stdout:778: finishing write tid 1 to smithi017107884-17 2023-03-29T14:05:50.317 INFO:tasks.rados.rados.0.smithi017.stdout:778: finishing write tid 2 to smithi017107884-17 2023-03-29T14:05:50.321 INFO:tasks.rados.rados.0.smithi017.stdout:778: finishing write tid 3 to smithi017107884-17 2023-03-29T14:05:50.330 INFO:tasks.rados.rados.0.smithi017.stdout:778: finishing write tid 4 to smithi017107884-17 2023-03-29T14:05:50.330 INFO:tasks.rados.rados.0.smithi017.stdout:778: oid 17 updating version 0 to 112 2023-03-29T14:05:50.330 INFO:tasks.rados.rados.0.smithi017.stdout:778: oid 17 version 112 is already newer than 112 2023-03-29T14:05:50.331 INFO:tasks.rados.rados.0.smithi017.stdout:778: oid 17 version 112 is already newer than 112 2023-03-29T14:05:50.331 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 112 (ObjNum 378 snap 92 seq_num 378) dirty exists 2023-03-29T14:05:50.331 INFO:tasks.rados.rados.0.smithi017.stdout:778: left oid 17 (ObjNum 378 snap 92 seq_num 378) 2023-03-29T14:05:50.333 INFO:tasks.rados.rados.0.smithi017.stdout:780: finishing write tid 1 to smithi017107884-16 2023-03-29T14:05:50.376 INFO:tasks.rados.rados.0.smithi017.stdout:779: finishing write tid 1 to smithi017107884-21 2023-03-29T14:05:50.384 INFO:tasks.rados.rados.0.smithi017.stdout:780: finishing write tid 2 to smithi017107884-16 2023-03-29T14:05:50.385 INFO:tasks.rados.rados.0.smithi017.stdout:780: finishing write tid 3 to smithi017107884-16 2023-03-29T14:05:50.386 INFO:tasks.rados.rados.0.smithi017.stdout:780: finishing write tid 4 to smithi017107884-16 2023-03-29T14:05:50.386 INFO:tasks.rados.rados.0.smithi017.stdout:780: oid 16 updating version 0 to 46 2023-03-29T14:05:50.386 INFO:tasks.rados.rados.0.smithi017.stdout:780: oid 16 updating version 46 to 49 2023-03-29T14:05:50.386 INFO:tasks.rados.rados.0.smithi017.stdout:780: oid 16 version 49 is already newer than 49 2023-03-29T14:05:50.386 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 49 (ObjNum 380 snap 92 seq_num 380) dirty exists 2023-03-29T14:05:50.386 INFO:tasks.rados.rados.0.smithi017.stdout:780: left oid 16 (ObjNum 380 snap 92 seq_num 380) 2023-03-29T14:05:50.387 INFO:tasks.rados.rados.0.smithi017.stdout:778: done (5 left) 2023-03-29T14:05:50.387 INFO:tasks.rados.rados.0.smithi017.stdout:780: done (4 left) 2023-03-29T14:05:50.387 INFO:tasks.rados.rados.0.smithi017.stdout:783: done (3 left) 2023-03-29T14:05:50.387 INFO:tasks.rados.rados.0.smithi017.stdout:784: delete oid 3 current snap is 92 2023-03-29T14:05:50.439 INFO:tasks.rados.rados.0.smithi017.stdout:779: finishing write tid 2 to smithi017107884-21 2023-03-29T14:05:50.451 INFO:tasks.rados.rados.0.smithi017.stdout:784: done (3 left) 2023-03-29T14:05:50.451 INFO:tasks.rados.rados.0.smithi017.stdout:785: delete oid 32 current snap is 92 2023-03-29T14:05:50.457 INFO:tasks.rados.rados.0.smithi017.stdout:779: finishing write tid 3 to smithi017107884-21 2023-03-29T14:05:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:779: finishing write tid 4 to smithi017107884-21 2023-03-29T14:05:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:779: oid 21 updating version 0 to 78 2023-03-29T14:05:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:779: oid 21 version 78 is already newer than 78 2023-03-29T14:05:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:779: oid 21 version 78 is already newer than 78 2023-03-29T14:05:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 78 (ObjNum 379 snap 92 seq_num 379) dirty exists 2023-03-29T14:05:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:779: left oid 21 (ObjNum 379 snap 92 seq_num 379) 2023-03-29T14:05:50.510 INFO:tasks.rados.rados.0.smithi017.stdout:779: done (3 left) 2023-03-29T14:05:50.511 INFO:tasks.rados.rados.0.smithi017.stdout:785: done (2 left) 2023-03-29T14:05:50.511 INFO:tasks.rados.rados.0.smithi017.stdout:786: read oid 26 snap 90 2023-03-29T14:05:50.511 INFO:tasks.rados.rados.0.smithi017.stdout:786: expect (ObjNum 365 snap 90 seq_num 365) 2023-03-29T14:05:50.511 INFO:tasks.rados.rados.0.smithi017.stdout:787: read oid 3 snap -1 2023-03-29T14:05:50.511 INFO:tasks.rados.rados.0.smithi017.stdout:787: expect deleted 2023-03-29T14:05:50.511 INFO:tasks.rados.rados.0.smithi017.stdout:788: snap_remove snap 83 2023-03-29T14:05:51.091 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:51.091 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:51.091 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:51.223 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:51.218+0000 7f3261cf7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:51.225+0000 7f3261cf7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:51.240 INFO:tasks.rados.rados.0.smithi017.stdout:788: done (4 left) 2023-03-29T14:05:51.240 INFO:tasks.rados.rados.0.smithi017.stdout:789: snap_create 2023-03-29T14:05:51.240 INFO:tasks.rados.rados.0.smithi017.stdout:786: expect (ObjNum 365 snap 90 seq_num 365) 2023-03-29T14:05:51.336 INFO:tasks.rados.rados.0.smithi017.stdout:782: finishing write tid 1 to smithi017107884-47 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:782: finishing write tid 2 to smithi017107884-47 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:781: finishing write tid 1 to smithi017107884-25 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:782: finishing write tid 3 to smithi017107884-47 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:782: finishing write tid 4 to smithi017107884-47 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:782: oid 47 updating version 0 to 126 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:782: oid 47 version 126 is already newer than 126 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:782: oid 47 version 126 is already newer than 126 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 126 (ObjNum 382 snap 92 seq_num 382) dirty exists 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:782: left oid 47 (ObjNum 382 snap 92 seq_num 382) 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:781: finishing write tid 2 to smithi017107884-25 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:781: finishing write tid 3 to smithi017107884-25 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:781: finishing write tid 4 to smithi017107884-25 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:781: oid 25 updating version 0 to 119 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:781: oid 25 version 119 is already newer than 116 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:781: oid 25 version 119 is already newer than 119 2023-03-29T14:05:51.337 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 119 (ObjNum 381 snap 92 seq_num 381) dirty exists 2023-03-29T14:05:51.338 INFO:tasks.rados.rados.0.smithi017.stdout:781: left oid 25 (ObjNum 381 snap 92 seq_num 381) 2023-03-29T14:05:51.431 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:51.445 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2023-03-29T14:05:51.239574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190026432,"num_objects":444,"num_object_clones":348,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":1831,"num_read_kb":1012283,"num_write":2494,"num_write_kb":653312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1298,"ondisk_log_size":1298,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":933776,"kb_used_data":880304,"kb_used_omap":0,"kb_used_meta":53440,"kb_avail":376553584,"statfs":{"total":386547056640,"available":385590870016,"internally_reserved":0,"allocated":901431296,"data_stored":1087432515,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":54722560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015676"},"pg_stats":[{"pgid":"3.f","version":"198'42","reported_seq":25,"reported_epoch":201,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:45.492847+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:05:45.492847+0000","last_peered":"2023-03-29T14:05:45.492847+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:05:45.492847+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":198,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:44.277283+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:05:44.277283+0000","last_peered":"2023-03-29T14:05:44.277283+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:05:44.277283+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"188'104","reported_seq":394,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-03-29T14:05:45.499661+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:45.499661+0000","last_peered":"2023-03-29T14:05:45.499661+0000","last_clean":"2023-03-29T14:05:45.499661+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:45.499661+0000","last_undegraded":"2023-03-29T14:05:45.499661+0000","last_fullsized":"2023-03-29T14:05:45.499661+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13949942,"num_objects":34,"num_object_clones":24,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":149,"num_read_kb":109409,"num_write":202,"num_write_kb":60172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"201'115","reported_seq":421,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-03-29T14:05:46.355572+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:46.355572+0000","last_peered":"2023-03-29T14:05:46.355572+0000","last_clean":"2023-03-29T14:05:46.355572+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:46.355572+0000","last_undegraded":"2023-03-29T14:05:46.355572+0000","last_fullsized":"2023-03-29T14:05:46.355572+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15866138,"num_objects":50,"num_object_clones":42,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":180,"num_read_kb":112828,"num_write":224,"num_write_kb":62950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"198'101","reported_seq":381,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-03-29T14:05:44.269306+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:44.269306+0000","last_peered":"2023-03-29T14:05:44.269306+0000","last_clean":"2023-03-29T14:05:44.269306+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:44.269306+0000","last_undegraded":"2023-03-29T14:05:44.269306+0000","last_fullsized":"2023-03-29T14:05:44.269306+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18924880,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":145,"num_read_kb":77953,"num_write":204,"num_write_kb":61526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":161,"reported_epoch":201,"state":"active+undersized","last_fresh":"2023-03-29T14:05:45.498034+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:05:45.498034+0000","last_peered":"2023-03-29T14:05:45.498034+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:05:45.498034+0000","last_undegraded":"2023-03-29T14:05:45.498034+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"191'78","reported_seq":333,"reported_epoch":201,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:45.494610+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:05:45.494610+0000","last_peered":"2023-03-29T14:05:45.494610+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:05:45.494610+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15361730,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":107,"num_read_kb":51656,"num_write":146,"num_write_kb":43166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":32}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'51","reported_seq":278,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-03-29T14:05:45.484177+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:45.484177+0000","last_peered":"2023-03-29T14:05:45.484177+0000","last_clean":"2023-03-29T14:05:45.484177+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:45.484177+0000","last_undegraded":"2023-03-29T14:05:45.484177+0000","last_fullsized":"2023-03-29T14:05:45.484177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":81,"num_read_kb":49140,"num_write":108,"num_write_kb":31002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":160,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-03-29T14:05:44.250696+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:05:44.250696+0000","last_peered":"2023-03-29T14:05:44.250696+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:05:44.250696+0000","last_undegraded":"2023-03-29T14:05:44.250696+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'37","reported_seq":22,"reported_epoch":201,"state":"active+undersized","last_fresh":"2023-03-29T14:05:42.475692+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:05:42.475692+0000","last_peered":"2023-03-29T14:05:42.475692+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:05:42.475692+0000","last_undegraded":"2023-03-29T14:05:42.475692+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":164,"reported_epoch":201,"state":"active+undersized","last_fresh":"2023-03-29T14:05:45.488141+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:05:45.488141+0000","last_peered":"2023-03-29T14:05:45.488141+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:05:45.488141+0000","last_undegraded":"2023-03-29T14:05:45.488141+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":286,"reported_epoch":201,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:42.493290+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:05:42.493290+0000","last_peered":"2023-03-29T14:05:42.493290+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:05:42.493290+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":46474,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-03-29T14:05:44.254660+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:05:44.254660+0000","last_peered":"2023-03-29T14:05:44.254660+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:05:44.254660+0000","last_undegraded":"2023-03-29T14:05:44.254660+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"196'81","reported_seq":337,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:44.274670+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:05:44.274670+0000","last_peered":"2023-03-29T14:05:44.274670+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:05:44.274670+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193792,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":45592,"num_write":170,"num_write_kb":51308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":201,"state":"active+undersized","last_fresh":"2023-03-29T14:05:45.482916+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:05:45.482916+0000","last_peered":"2023-03-29T14:05:45.482916+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:05:45.482916+0000","last_undegraded":"2023-03-29T14:05:45.482916+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":385,"reported_epoch":200,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:44.237332+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:05:44.237332+0000","last_peered":"2023-03-29T14:05:44.237332+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:05:44.237332+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"201'71","reported_seq":350,"reported_epoch":201,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:46.257492+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:05:46.257492+0000","last_peered":"2023-03-29T14:05:46.257492+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:05:46.257492+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15529800,"num_objects":28,"num_object_clones":20,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":103516,"num_write":142,"num_write_kb":40232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":164,"reported_epoch":201,"state":"active+undersized","last_fresh":"2023-03-29T14:05:45.489420+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:05:45.489420+0000","last_peered":"2023-03-29T14:05:45.489420+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:05:45.489420+0000","last_undegraded":"2023-03-29T14:05:45.489420+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"198'162","reported_seq":538,"reported_epoch":201,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:43.165385+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:05:43.165385+0000","last_peered":"2023-03-29T14:05:43.165385+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:05:43.165385+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22790622,"num_objects":60,"num_object_clones":46,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":270,"num_read_kb":174546,"num_write":310,"num_write_kb":89344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-03-29T14:05:44.278692+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:05:44.278692+0000","last_peered":"2023-03-29T14:05:44.278692+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:05:44.278692+0000","last_undegraded":"2023-03-29T14:05:44.278692+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"198'60","reported_seq":289,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-03-29T14:05:44.260475+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:44.260475+0000","last_peered":"2023-03-29T14:05:44.260475+0000","last_clean":"2023-03-29T14:05:44.260475+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:44.260475+0000","last_undegraded":"2023-03-29T14:05:44.260475+0000","last_fullsized":"2023-03-29T14:05:44.260475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8202080,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":47065,"num_write":126,"num_write_kb":35686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"195'118","reported_seq":411,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-03-29T14:05:44.243568+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:44.243568+0000","last_peered":"2023-03-29T14:05:44.243568+0000","last_clean":"2023-03-29T14:05:44.243568+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:44.243568+0000","last_undegraded":"2023-03-29T14:05:44.243568+0000","last_fullsized":"2023-03-29T14:05:44.243568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12773452,"num_objects":48,"num_object_clones":38,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":160,"num_read_kb":92205,"num_write":226,"num_write_kb":68272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":160,"reported_epoch":200,"state":"active+undersized","last_fresh":"2023-03-29T14:05:44.258672+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:05:44.258672+0000","last_peered":"2023-03-29T14:05:44.258672+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:05:44.258672+0000","last_undegraded":"2023-03-29T14:05:44.258672+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":201,"state":"active+undersized","last_fresh":"2023-03-29T14:05:45.495950+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:05:45.495950+0000","last_peered":"2023-03-29T14:05:45.495950+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:05:45.495950+0000","last_undegraded":"2023-03-29T14:05:45.495950+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"192'106","reported_seq":394,"reported_epoch":201,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:45.612702+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:05:45.612702+0000","last_peered":"2023-03-29T14:05:45.612702+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:05:45.612702+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21008948,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":148,"num_read_kb":83789,"num_write":206,"num_write_kb":60894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":188845696,"num_objects":440,"num_object_clones":348,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":1689,"num_read_kb":1012163,"num_write":2220,"num_write_kb":649524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1201,"ondisk_log_size":1201,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":242324,"kb_used_data":229068,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94129516,"statfs":{"total":96636764160,"available":96388624384,"internally_reserved":0,"allocated":234565632,"data_stored":281243100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":276884,"kb_used_data":262668,"kb_used_omap":0,"kb_used_meta":14208,"kb_avail":94094956,"statfs":{"total":96636764160,"available":96353234944,"internally_reserved":0,"allocated":268972032,"data_stored":327240845,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14548992},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":258624,"kb_used_data":244728,"kb_used_omap":0,"kb_used_meta":13888,"kb_avail":94113216,"statfs":{"total":96636764160,"available":96371933184,"internally_reserved":0,"allocated":250601472,"data_stored":305716345,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14221312},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:51.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.0 1.0 3.2 3.1 3.6 2023-03-29T14:05:51.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:51.576+0000 7f54b7ba3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:51.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:51.588+0000 7f54b7ba3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:51.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:51.782 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:51.780+0000 7fe1044c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:51.786 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:51.785+0000 7fe1044c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.6] on osd.0 to force-recovery; instructing pg(s) [1.0,3.0] on osd.1 to force-recovery; instructing pg(s) [3.2] on osd.3 to force-recovery; 2023-03-29T14:05:51.795 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:05:51.852 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:51.856 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:51.856 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:52.242 INFO:tasks.rados.rados.0.smithi017.stdout:781: done (4 left) 2023-03-29T14:05:52.242 INFO:tasks.rados.rados.0.smithi017.stdout:782: done (3 left) 2023-03-29T14:05:52.242 INFO:tasks.rados.rados.0.smithi017.stdout:786: done (2 left) 2023-03-29T14:05:52.242 INFO:tasks.rados.rados.0.smithi017.stdout:787: done (1 left) 2023-03-29T14:05:52.242 INFO:tasks.rados.rados.0.smithi017.stdout:789: done (0 left) 2023-03-29T14:05:52.242 INFO:tasks.rados.rados.0.smithi017.stdout:790: snap_remove snap 92 2023-03-29T14:05:53.213 INFO:tasks.rados.rados.0.smithi017.stdout:790: done (0 left) 2023-03-29T14:05:53.213 INFO:tasks.rados.rados.0.smithi017.stdout:791: write oid 9 current snap is 93 2023-03-29T14:05:53.213 INFO:tasks.rados.rados.0.smithi017.stdout:791: seq_num 386 ranges {487502=463764,1738167=446469} 2023-03-29T14:05:53.218 INFO:tasks.rados.rados.0.smithi017.stdout:791: writing smithi017107884-9 from 487502 to 951266 tid 1 2023-03-29T14:05:53.225 INFO:tasks.rados.rados.0.smithi017.stdout:791: writing smithi017107884-9 from 1738167 to 2184636 tid 2 2023-03-29T14:05:53.226 INFO:tasks.rados.rados.0.smithi017.stdout:792: read oid 33 snap -1 2023-03-29T14:05:53.226 INFO:tasks.rados.rados.0.smithi017.stdout:792: expect (ObjNum 288 snap 76 seq_num 288) 2023-03-29T14:05:53.226 INFO:tasks.rados.rados.0.smithi017.stdout:793: read oid 31 snap -1 2023-03-29T14:05:53.226 INFO:tasks.rados.rados.0.smithi017.stdout:793: expect deleted 2023-03-29T14:05:53.226 INFO:tasks.rados.rados.0.smithi017.stdout:794: write oid 29 current snap is 93 2023-03-29T14:05:53.226 INFO:tasks.rados.rados.0.smithi017.stdout:794: seq_num 387 ranges {504309=539013,1796649=590127} 2023-03-29T14:05:53.230 INFO:tasks.rados.rados.0.smithi017.stdout:794: writing smithi017107884-29 from 504309 to 1043322 tid 1 2023-03-29T14:05:53.238 INFO:tasks.rados.rados.0.smithi017.stdout:794: writing smithi017107884-29 from 1796649 to 2386776 tid 2 2023-03-29T14:05:53.238 INFO:tasks.rados.rados.0.smithi017.stdout:795: delete oid 48 current snap is 93 2023-03-29T14:05:53.279 INFO:tasks.rados.rados.0.smithi017.stdout:793: done (4 left) 2023-03-29T14:05:53.279 INFO:tasks.rados.rados.0.smithi017.stdout:795: done (3 left) 2023-03-29T14:05:53.279 INFO:tasks.rados.rados.0.smithi017.stdout:796: snap_remove snap 90 2023-03-29T14:05:53.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:53.983+0000 7fd3f2a5e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:53.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:53.988+0000 7fd3f2a5e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:54.053 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:54.056 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:54.057 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:54.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~44s 2023-03-29T14:05:54.248 INFO:tasks.rados.rados.0.smithi017.stdout:796: done (3 left) 2023-03-29T14:05:54.248 INFO:tasks.rados.rados.0.smithi017.stdout:797: read oid 42 snap -1 2023-03-29T14:05:54.248 INFO:tasks.rados.rados.0.smithi017.stdout:797: expect deleted 2023-03-29T14:05:54.249 INFO:tasks.rados.rados.0.smithi017.stdout:798: write oid 38 current snap is 93 2023-03-29T14:05:54.249 INFO:tasks.rados.rados.0.smithi017.stdout:798: seq_num 389 ranges {537923=405864,1513613=593796} 2023-03-29T14:05:54.253 INFO:tasks.rados.rados.0.smithi017.stdout:798: writing smithi017107884-38 from 537923 to 943787 tid 1 2023-03-29T14:05:54.260 INFO:tasks.rados.rados.0.smithi017.stdout:798: writing smithi017107884-38 from 1513613 to 2107409 tid 2 2023-03-29T14:05:54.260 INFO:tasks.rados.rados.0.smithi017.stdout:799: write (excl) oid 12 current snap is 93 2023-03-29T14:05:54.260 INFO:tasks.rados.rados.0.smithi017.stdout:799: seq_num 390 ranges {554730=481113,1572095=737454} 2023-03-29T14:05:54.265 INFO:tasks.rados.rados.0.smithi017.stdout:799: writing smithi017107884-12 from 554730 to 1035843 tid 1 2023-03-29T14:05:54.273 INFO:tasks.rados.rados.0.smithi017.stdout:799: writing smithi017107884-12 from 1572095 to 2309549 tid 2 2023-03-29T14:05:54.273 INFO:tasks.rados.rados.0.smithi017.stdout:800: delete oid 7 current snap is 93 2023-03-29T14:05:54.273 INFO:tasks.rados.rados.0.smithi017.stdout:792: expect (ObjNum 288 snap 76 seq_num 288) 2023-03-29T14:05:54.468 INFO:tasks.rados.rados.0.smithi017.stdout:791: finishing write tid 1 to smithi017107884-9 2023-03-29T14:05:54.468 INFO:tasks.rados.rados.0.smithi017.stdout:794: finishing write tid 1 to smithi017107884-29 2023-03-29T14:05:54.468 INFO:tasks.rados.rados.0.smithi017.stdout:791: finishing write tid 2 to smithi017107884-9 2023-03-29T14:05:54.468 INFO:tasks.rados.rados.0.smithi017.stdout:791: finishing write tid 3 to smithi017107884-9 2023-03-29T14:05:54.468 INFO:tasks.rados.rados.0.smithi017.stdout:791: finishing write tid 4 to smithi017107884-9 2023-03-29T14:05:54.468 INFO:tasks.rados.rados.0.smithi017.stdout:791: oid 9 updating version 0 to 123 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:791: oid 9 version 123 is already newer than 123 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:791: oid 9 version 123 is already newer than 123 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 123 (ObjNum 386 snap 93 seq_num 386) dirty exists 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:791: left oid 9 (ObjNum 386 snap 93 seq_num 386) 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: finishing write tid 2 to smithi017107884-29 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: finishing write tid 3 to smithi017107884-29 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: finishing write tid 4 to smithi017107884-29 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: oid 29 updating version 0 to 62 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: oid 29 version 62 is already newer than 62 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: oid 29 version 62 is already newer than 62 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 62 (ObjNum 387 snap 93 seq_num 387) dirty exists 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: left oid 29 (ObjNum 387 snap 93 seq_num 387) 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:791: done (6 left) 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:792: done (5 left) 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:794: done (4 left) 2023-03-29T14:05:54.469 INFO:tasks.rados.rados.0.smithi017.stdout:800: done (3 left) 2023-03-29T14:05:54.470 INFO:tasks.rados.rados.0.smithi017.stdout:801: read oid 34 snap -1 2023-03-29T14:05:54.470 INFO:tasks.rados.rados.0.smithi017.stdout:801: expect (ObjNum 344 snap 82 seq_num 344) 2023-03-29T14:05:54.470 INFO:tasks.rados.rados.0.smithi017.stdout:802: snap_remove snap 91 2023-03-29T14:05:55.250 INFO:tasks.rados.rados.0.smithi017.stdout:802: done (4 left) 2023-03-29T14:05:55.251 INFO:tasks.rados.rados.0.smithi017.stdout:803: read oid 41 snap -1 2023-03-29T14:05:55.251 INFO:tasks.rados.rados.0.smithi017.stdout:803: expect deleted 2023-03-29T14:05:55.251 INFO:tasks.rados.rados.0.smithi017.stdout:804: read oid 6 snap -1 2023-03-29T14:05:55.252 INFO:tasks.rados.rados.0.smithi017.stdout:804: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:05:55.252 INFO:tasks.rados.rados.0.smithi017.stdout:805: read oid 29 snap -1 2023-03-29T14:05:55.252 INFO:tasks.rados.rados.0.smithi017.stdout:805: expect (ObjNum 387 snap 93 seq_num 387) 2023-03-29T14:05:55.252 INFO:tasks.rados.rados.0.smithi017.stdout:806: snap_create 2023-03-29T14:05:55.252 INFO:tasks.rados.rados.0.smithi017.stdout:798: finishing write tid 1 to smithi017107884-38 2023-03-29T14:05:55.253 INFO:tasks.rados.rados.0.smithi017.stdout:798: finishing write tid 2 to smithi017107884-38 2023-03-29T14:05:55.253 INFO:tasks.rados.rados.0.smithi017.stdout:798: finishing write tid 3 to smithi017107884-38 2023-03-29T14:05:55.253 INFO:tasks.rados.rados.0.smithi017.stdout:798: finishing write tid 4 to smithi017107884-38 2023-03-29T14:05:55.253 INFO:tasks.rados.rados.0.smithi017.stdout:798: oid 38 updating version 0 to 21 2023-03-29T14:05:55.254 INFO:tasks.rados.rados.0.smithi017.stdout:798: oid 38 version 21 is already newer than 21 2023-03-29T14:05:55.254 INFO:tasks.rados.rados.0.smithi017.stdout:798: oid 38 version 21 is already newer than 21 2023-03-29T14:05:55.254 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 21 (ObjNum 389 snap 93 seq_num 389) dirty exists 2023-03-29T14:05:55.254 INFO:tasks.rados.rados.0.smithi017.stdout:798: left oid 38 (ObjNum 389 snap 93 seq_num 389) 2023-03-29T14:05:55.254 INFO:tasks.rados.rados.0.smithi017.stdout:799: finishing write tid 1 to smithi017107884-12 2023-03-29T14:05:55.255 INFO:tasks.rados.rados.0.smithi017.stdout:799: finishing write tid 2 to smithi017107884-12 2023-03-29T14:05:55.255 INFO:tasks.rados.rados.0.smithi017.stdout:799: finishing write tid 3 to smithi017107884-12 2023-03-29T14:05:55.255 INFO:tasks.rados.rados.0.smithi017.stdout:799: finishing write tid 4 to smithi017107884-12 2023-03-29T14:05:55.255 INFO:tasks.rados.rados.0.smithi017.stdout:799: oid 12 updating version 0 to 130 2023-03-29T14:05:55.255 INFO:tasks.rados.rados.0.smithi017.stdout:799: oid 12 version 130 is already newer than 130 2023-03-29T14:05:55.256 INFO:tasks.rados.rados.0.smithi017.stdout:799: oid 12 version 130 is already newer than 127 2023-03-29T14:05:55.256 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 130 (ObjNum 390 snap 93 seq_num 390) dirty exists 2023-03-29T14:05:55.256 INFO:tasks.rados.rados.0.smithi017.stdout:799: left oid 12 (ObjNum 390 snap 93 seq_num 390) 2023-03-29T14:05:55.256 INFO:tasks.rados.rados.0.smithi017.stdout:801: expect (ObjNum 344 snap 82 seq_num 344) 2023-03-29T14:05:55.566 INFO:tasks.rados.rados.0.smithi017.stdout:804: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:05:55.796 INFO:tasks.rados.rados.0.smithi017.stdout:805: expect (ObjNum 387 snap 93 seq_num 387) 2023-03-29T14:05:56.058 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:05:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:56.185+0000 7fd5a3b55700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:56.189+0000 7fd5a3b55700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:56.255 INFO:tasks.rados.rados.0.smithi017.stdout:797: done (7 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:798: done (6 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:799: done (5 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:801: done (4 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:803: done (3 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:804: done (2 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:805: done (1 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:806: done (0 left) 2023-03-29T14:05:56.256 INFO:tasks.rados.rados.0.smithi017.stdout:807: delete oid 11 current snap is 94 2023-03-29T14:05:56.258 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:56.261 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:56.262 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:56.367 INFO:tasks.rados.rados.0.smithi017.stdout:807: done (0 left) 2023-03-29T14:05:56.367 INFO:tasks.rados.rados.0.smithi017.stdout:808: snap_remove snap 93 2023-03-29T14:05:56.808 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:05:56.808 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:05:56.808 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:05:56.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:56.935+0000 7f587b6f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:56.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:56.941+0000 7f587b6f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:57.151 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:05:57.153 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:05:57.164 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2023-03-29T14:05:56.254469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203439408,"num_objects":460,"num_object_clones":364,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1895,"num_read_kb":1034199,"num_write":2606,"num_write_kb":680236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1353,"ondisk_log_size":1353,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":967868,"kb_used_data":913264,"kb_used_omap":0,"kb_used_meta":54016,"kb_avail":376519492,"statfs":{"total":386547056640,"available":385555959808,"internally_reserved":0,"allocated":935182336,"data_stored":1123565452,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":55312384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014260"},"pg_stats":[{"pgid":"3.f","version":"205'49","reported_seq":45,"reported_epoch":206,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:50.386942+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:05:50.386942+0000","last_peered":"2023-03-29T14:05:50.386942+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:05:50.386942+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4773320,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":16,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"176'17","reported_seq":202,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:48.284451+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:05:48.284451+0000","last_peered":"2023-03-29T14:05:48.284451+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:05:48.284451+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17990,"num_write":36,"num_write_kb":11038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"205'112","reported_seq":413,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-03-29T14:05:50.325994+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:05:50.325994+0000","last_peered":"2023-03-29T14:05:50.325994+0000","last_clean":"2023-03-29T14:05:50.325994+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:05:50.325994+0000","last_undegraded":"2023-03-29T14:05:50.325994+0000","last_fullsized":"2023-03-29T14:05:50.325994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9210500,"num_objects":38,"num_object_clones":28,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":153,"num_read_kb":109411,"num_write":218,"num_write_kb":64190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"205'119","reported_seq":433,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-03-29T14:05:50.586955+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:05:50.586955+0000","last_peered":"2023-03-29T14:05:50.586955+0000","last_clean":"2023-03-29T14:05:50.586955+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:05:50.586955+0000","last_undegraded":"2023-03-29T14:05:50.586955+0000","last_fullsized":"2023-03-29T14:05:50.586955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16538418,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":183,"num_read_kb":112829,"num_write":232,"num_write_kb":65036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"198'101","reported_seq":385,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-03-29T14:05:48.319439+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:48.319439+0000","last_peered":"2023-03-29T14:05:48.319439+0000","last_clean":"2023-03-29T14:05:48.319439+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:48.319439+0000","last_undegraded":"2023-03-29T14:05:48.319439+0000","last_fullsized":"2023-03-29T14:05:48.319439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18924880,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":145,"num_read_kb":77953,"num_write":204,"num_write_kb":61526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":166,"reported_epoch":206,"state":"active+undersized","last_fresh":"2023-03-29T14:05:50.250883+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:05:50.250883+0000","last_peered":"2023-03-29T14:05:50.250883+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:05:50.250883+0000","last_undegraded":"2023-03-29T14:05:50.250883+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"204'84","reported_seq":347,"reported_epoch":206,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:50.254111+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:05:50.254111+0000","last_peered":"2023-03-29T14:05:50.254111+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:05:50.254111+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15361730,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":108,"num_read_kb":51657,"num_write":154,"num_write_kb":45076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":34}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"202'55","reported_seq":290,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-03-29T14:05:50.257600+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:05:50.257600+0000","last_peered":"2023-03-29T14:05:50.257600+0000","last_clean":"2023-03-29T14:05:50.257600+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:05:50.257600+0000","last_undegraded":"2023-03-29T14:05:50.257600+0000","last_fullsized":"2023-03-29T14:05:50.257600+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10689384,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":49141,"num_write":116,"num_write_kb":32686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-03-29T14:05:48.296530+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:05:48.296530+0000","last_peered":"2023-03-29T14:05:48.296530+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:05:48.296530+0000","last_undegraded":"2023-03-29T14:05:48.296530+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":33,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-03-29T14:05:49.306412+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:05:49.306412+0000","last_peered":"2023-03-29T14:05:49.306412+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:05:49.306412+0000","last_undegraded":"2023-03-29T14:05:49.306412+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":206,"state":"active+undersized","last_fresh":"2023-03-29T14:05:50.258304+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:05:50.258304+0000","last_peered":"2023-03-29T14:05:50.258304+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:05:50.258304+0000","last_undegraded":"2023-03-29T14:05:50.258304+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"164'58","reported_seq":293,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:49.273706+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:05:49.273706+0000","last_peered":"2023-03-29T14:05:49.273706+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:05:49.273706+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11798646,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":52860,"num_write":120,"num_write_kb":33934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-03-29T14:05:48.325672+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:05:48.325672+0000","last_peered":"2023-03-29T14:05:48.325672+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:05:48.325672+0000","last_undegraded":"2023-03-29T14:05:48.325672+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"196'81","reported_seq":341,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:48.323253+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:05:48.323253+0000","last_peered":"2023-03-29T14:05:48.323253+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:05:48.323253+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193792,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":45592,"num_write":170,"num_write_kb":51308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":206,"state":"active+undersized","last_fresh":"2023-03-29T14:05:50.247145+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:05:50.247145+0000","last_peered":"2023-03-29T14:05:50.247145+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:05:50.247145+0000","last_undegraded":"2023-03-29T14:05:50.247145+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":389,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:48.293596+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:05:48.293596+0000","last_peered":"2023-03-29T14:05:48.293596+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:05:48.293596+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"205'78","reported_seq":372,"reported_epoch":206,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:50.522192+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:05:50.522192+0000","last_peered":"2023-03-29T14:05:50.522192+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:05:50.522192+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20202476,"num_objects":30,"num_object_clones":22,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":108788,"num_write":158,"num_write_kb":44220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":169,"reported_epoch":206,"state":"active+undersized","last_fresh":"2023-03-29T14:05:50.252920+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:05:50.252920+0000","last_peered":"2023-03-29T14:05:50.252920+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:05:50.252920+0000","last_undegraded":"2023-03-29T14:05:50.252920+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"202'165","reported_seq":552,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:50.456834+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:05:50.456834+0000","last_peered":"2023-03-29T14:05:50.456834+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:05:50.456834+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740036,"num_objects":62,"num_object_clones":48,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":62,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":276,"num_read_kb":174547,"num_write":318,"num_write_kb":90962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":62}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-03-29T14:05:48.292305+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:05:48.292305+0000","last_peered":"2023-03-29T14:05:48.292305+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:05:48.292305+0000","last_undegraded":"2023-03-29T14:05:48.292305+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"202'63","reported_seq":303,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-03-29T14:05:48.385687+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:48.385687+0000","last_peered":"2023-03-29T14:05:48.385687+0000","last_clean":"2023-03-29T14:05:48.385687+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:48.385687+0000","last_undegraded":"2023-03-29T14:05:48.385687+0000","last_fullsized":"2023-03-29T14:05:48.385687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8639062,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":51518,"num_write":134,"num_write_kb":37726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"205'126","reported_seq":433,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-03-29T14:05:50.473469+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:50.473469+0000","last_peered":"2023-03-29T14:05:50.473469+0000","last_clean":"2023-03-29T14:05:50.473469+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:50.473469+0000","last_undegraded":"2023-03-29T14:05:50.473469+0000","last_fullsized":"2023-03-29T14:05:50.473469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412382,"num_objects":50,"num_object_clones":40,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":169,"num_read_kb":98001,"num_write":242,"num_write_kb":72046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+undersized","last_fresh":"2023-03-29T14:05:48.313521+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:05:48.313521+0000","last_peered":"2023-03-29T14:05:48.313521+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:05:48.313521+0000","last_undegraded":"2023-03-29T14:05:48.313521+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":169,"reported_epoch":206,"state":"active+undersized","last_fresh":"2023-03-29T14:05:50.252057+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:05:50.252057+0000","last_peered":"2023-03-29T14:05:50.252057+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:05:50.252057+0000","last_undegraded":"2023-03-29T14:05:50.252057+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"192'106","reported_seq":397,"reported_epoch":205,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:49.304540+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:05:49.304540+0000","last_peered":"2023-03-29T14:05:49.304540+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:05:49.304540+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21008948,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":148,"num_read_kb":83789,"num_write":206,"num_write_kb":60894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":202258672,"num_objects":456,"num_object_clones":364,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":1753,"num_read_kb":1034079,"num_write":2332,"num_write_kb":676448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1256,"ondisk_log_size":1256,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":255980,"kb_used_data":242532,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94115860,"statfs":{"total":96636764160,"available":96374640640,"internally_reserved":0,"allocated":248352768,"data_stored":294743147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":286728,"kb_used_data":272320,"kb_used_omap":0,"kb_used_meta":14400,"kb_avail":94085112,"statfs":{"total":96636764160,"available":96343154688,"internally_reserved":0,"allocated":278855680,"data_stored":338806013,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14745600},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":269216,"kb_used_data":254572,"kb_used_omap":0,"kb_used_meta":14080,"kb_avail":94102624,"statfs":{"total":96636764160,"available":96361086976,"internally_reserved":0,"allocated":260681728,"data_stored":316784067,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14417920},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:05:57.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.9 1.0 2023-03-29T14:05:57.260 INFO:tasks.rados.rados.0.smithi017.stdout:808: done (0 left) 2023-03-29T14:05:57.261 INFO:tasks.rados.rados.0.smithi017.stdout:809: read oid 44 snap -1 2023-03-29T14:05:57.261 INFO:tasks.rados.rados.0.smithi017.stdout:809: expect (ObjNum 362 snap 89 seq_num 362) 2023-03-29T14:05:57.261 INFO:tasks.rados.rados.0.smithi017.stdout:810: write oid 16 current snap is 94 2023-03-29T14:05:57.261 INFO:tasks.rados.rados.0.smithi017.stdout:810: seq_num 393 ranges {605151=706860,2031188=573963} 2023-03-29T14:05:57.267 INFO:tasks.rados.rados.0.smithi017.stdout:810: writing smithi017107884-16 from 605151 to 1312011 tid 1 2023-03-29T14:05:57.275 INFO:tasks.rados.rados.0.smithi017.stdout:810: writing smithi017107884-16 from 2031188 to 2605151 tid 2 2023-03-29T14:05:57.275 INFO:tasks.rados.rados.0.smithi017.stdout:811: read oid 24 snap -1 2023-03-29T14:05:57.275 INFO:tasks.rados.rados.0.smithi017.stdout:811: expect (ObjNum 296 snap 78 seq_num 296) 2023-03-29T14:05:57.275 INFO:tasks.rados.rados.0.smithi017.stdout:812: write oid 19 current snap is 94 2023-03-29T14:05:57.275 INFO:tasks.rados.rados.0.smithi017.stdout:812: seq_num 394 ranges {621958=782109,2089670=532288} 2023-03-29T14:05:57.283 INFO:tasks.rados.rados.0.smithi017.stdout:812: writing smithi017107884-19 from 621958 to 1404067 tid 1 2023-03-29T14:05:57.290 INFO:tasks.rados.rados.0.smithi017.stdout:812: writing smithi017107884-19 from 2089670 to 2621958 tid 2 2023-03-29T14:05:57.291 INFO:tasks.rados.rados.0.smithi017.stdout:813: read oid 42 snap -1 2023-03-29T14:05:57.291 INFO:tasks.rados.rados.0.smithi017.stdout:813: expect deleted 2023-03-29T14:05:57.291 INFO:tasks.rados.rados.0.smithi017.stdout:814: write (excl) oid 11 current snap is 94 2023-03-29T14:05:57.291 INFO:tasks.rados.rados.0.smithi017.stdout:814: seq_num 395 ranges {638765=457358,1748152=604803} 2023-03-29T14:05:57.296 INFO:tasks.rados.rados.0.smithi017.stdout:814: writing smithi017107884-11 from 638765 to 1096123 tid 1 2023-03-29T14:05:57.303 INFO:tasks.rados.rados.0.smithi017.stdout:814: writing smithi017107884-11 from 1748152 to 2352955 tid 2 2023-03-29T14:05:57.303 INFO:tasks.rados.rados.0.smithi017.stdout:815: delete oid 13 current snap is 94 2023-03-29T14:05:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:57.298+0000 7f6c161fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:57.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:57.308+0000 7f6c161fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:57.395 INFO:tasks.rados.rados.0.smithi017.stdout:810: finishing write tid 1 to smithi017107884-16 2023-03-29T14:05:57.404 INFO:tasks.rados.rados.0.smithi017.stdout:809: expect (ObjNum 362 snap 89 seq_num 362) 2023-03-29T14:05:57.515 INFO:tasks.rados.rados.0.smithi017.stdout:810: finishing write tid 2 to smithi017107884-16 2023-03-29T14:05:57.515 INFO:tasks.rados.rados.0.smithi017.stdout:809: done (6 left) 2023-03-29T14:05:57.515 INFO:tasks.rados.rados.0.smithi017.stdout:813: done (5 left) 2023-03-29T14:05:57.515 INFO:tasks.rados.rados.0.smithi017.stdout:815: done (4 left) 2023-03-29T14:05:57.515 INFO:tasks.rados.rados.0.smithi017.stdout:816: delete oid 36 current snap is 94 2023-03-29T14:05:57.515 INFO:tasks.rados.rados.0.smithi017.stdout:811: expect (ObjNum 296 snap 78 seq_num 296) 2023-03-29T14:05:57.537 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [1.0] on osd.1 to force-recovery; instructing pg(s) [3.9,3.f] on osd.3 to force-recovery; 2023-03-29T14:05:57.537 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:05:57.724 INFO:tasks.rados.rados.0.smithi017.stdout:812: finishing write tid 1 to smithi017107884-19 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:812: finishing write tid 2 to smithi017107884-19 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:810: finishing write tid 3 to smithi017107884-16 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:810: finishing write tid 4 to smithi017107884-16 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:810: oid 16 updating version 0 to 53 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:810: oid 16 version 53 is already newer than 53 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:810: oid 16 version 53 is already newer than 53 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 53 (ObjNum 393 snap 94 seq_num 393) dirty exists 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:810: left oid 16 (ObjNum 393 snap 94 seq_num 393) 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:812: finishing write tid 3 to smithi017107884-19 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:812: finishing write tid 4 to smithi017107884-19 2023-03-29T14:05:57.725 INFO:tasks.rados.rados.0.smithi017.stdout:812: oid 19 updating version 0 to 109 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:812: oid 19 version 109 is already newer than 109 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:812: oid 19 version 109 is already newer than 109 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 109 (ObjNum 394 snap 94 seq_num 394) dirty exists 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:812: left oid 19 (ObjNum 394 snap 94 seq_num 394) 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:810: done (4 left) 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:811: done (3 left) 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:812: done (2 left) 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:816: done (1 left) 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:817: read oid 9 snap -1 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:817: expect (ObjNum 386 snap 93 seq_num 386) 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:818: delete oid 32 current snap is 94 2023-03-29T14:05:57.726 INFO:tasks.rados.rados.0.smithi017.stdout:814: finishing write tid 1 to smithi017107884-11 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:814: finishing write tid 2 to smithi017107884-11 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:814: finishing write tid 3 to smithi017107884-11 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:814: finishing write tid 4 to smithi017107884-11 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:814: oid 11 updating version 0 to 173 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:814: oid 11 version 173 is already newer than 171 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:814: oid 11 version 173 is already newer than 173 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 11 v 173 (ObjNum 395 snap 94 seq_num 395) dirty exists 2023-03-29T14:05:57.727 INFO:tasks.rados.rados.0.smithi017.stdout:814: left oid 11 (ObjNum 395 snap 94 seq_num 395) 2023-03-29T14:05:57.728 INFO:tasks.rados.rados.0.smithi017.stdout:814: done (2 left) 2023-03-29T14:05:57.729 INFO:tasks.rados.rados.0.smithi017.stdout:818: done (1 left) 2023-03-29T14:05:57.729 INFO:tasks.rados.rados.0.smithi017.stdout:819: read oid 15 snap -1 2023-03-29T14:05:57.729 INFO:tasks.rados.rados.0.smithi017.stdout:819: expect (ObjNum 373 snap 90 seq_num 373) 2023-03-29T14:05:57.729 INFO:tasks.rados.rados.0.smithi017.stdout:820: write (excl) oid 22 current snap is 94 2023-03-29T14:05:57.729 INFO:tasks.rados.rados.0.smithi017.stdout:820: seq_num 399 ranges {705993=474707,1582080=612141} 2023-03-29T14:05:57.735 INFO:tasks.rados.rados.0.smithi017.stdout:820: writing smithi017107884-22 from 705993 to 1180700 tid 1 2023-03-29T14:05:57.741 INFO:tasks.rados.rados.0.smithi017.stdout:820: writing smithi017107884-22 from 1582080 to 2194221 tid 2 2023-03-29T14:05:57.741 INFO:tasks.rados.rados.0.smithi017.stdout:821: snap_create 2023-03-29T14:05:57.741 INFO:tasks.rados.rados.0.smithi017.stdout:817: expect (ObjNum 386 snap 93 seq_num 386) 2023-03-29T14:05:58.024 INFO:tasks.rados.rados.0.smithi017.stdout:819: expect (ObjNum 373 snap 90 seq_num 373) 2023-03-29T14:05:58.141 INFO:tasks.rados.rados.0.smithi017.stdout:820: finishing write tid 1 to smithi017107884-22 2023-03-29T14:05:58.141 INFO:tasks.rados.rados.0.smithi017.stdout:820: finishing write tid 2 to smithi017107884-22 2023-03-29T14:05:58.141 INFO:tasks.rados.rados.0.smithi017.stdout:820: finishing write tid 3 to smithi017107884-22 2023-03-29T14:05:58.141 INFO:tasks.rados.rados.0.smithi017.stdout:820: finishing write tid 4 to smithi017107884-22 2023-03-29T14:05:58.141 INFO:tasks.rados.rados.0.smithi017.stdout:820: oid 22 updating version 0 to 54 2023-03-29T14:05:58.141 INFO:tasks.rados.rados.0.smithi017.stdout:820: oid 22 updating version 54 to 56 2023-03-29T14:05:58.141 INFO:tasks.rados.rados.0.smithi017.stdout:820: oid 22 version 56 is already newer than 56 2023-03-29T14:05:58.142 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 56 (ObjNum 399 snap 94 seq_num 399) dirty exists 2023-03-29T14:05:58.142 INFO:tasks.rados.rados.0.smithi017.stdout:820: left oid 22 (ObjNum 399 snap 94 seq_num 399) 2023-03-29T14:05:58.262 INFO:tasks.rados.rados.0.smithi017.stdout:817: done (3 left) 2023-03-29T14:05:58.263 INFO:tasks.rados.rados.0.smithi017.stdout:819: done (2 left) 2023-03-29T14:05:58.263 INFO:tasks.rados.rados.0.smithi017.stdout:820: done (1 left) 2023-03-29T14:05:58.263 INFO:tasks.rados.rados.0.smithi017.stdout:821: done (0 left) 2023-03-29T14:05:58.263 INFO:tasks.rados.rados.0.smithi017.stdout:822: delete oid 23 current snap is 95 2023-03-29T14:05:58.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:05:58.347 INFO:tasks.rados.rados.0.smithi017.stdout:822: done (0 left) 2023-03-29T14:05:58.347 INFO:tasks.rados.rados.0.smithi017.stdout:823: delete oid 27 current snap is 95 2023-03-29T14:05:58.389 INFO:tasks.rados.rados.0.smithi017.stdout:823: done (0 left) 2023-03-29T14:05:58.389 INFO:tasks.rados.rados.0.smithi017.stdout:824: delete oid 18 current snap is 95 2023-03-29T14:05:58.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:58.392+0000 7f9565e1c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:58.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:05:58.396+0000 7f9565e1c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:05:58.436 INFO:tasks.rados.rados.0.smithi017.stdout:824: done (0 left) 2023-03-29T14:05:58.436 INFO:tasks.rados.rados.0.smithi017.stdout:825: delete oid 39 current snap is 95 2023-03-29T14:05:58.437 INFO:tasks.rados.rados.0.smithi017.stdout:825: done (0 left) 2023-03-29T14:05:58.437 INFO:tasks.rados.rados.0.smithi017.stdout:826: snap_remove snap 94 2023-03-29T14:05:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:05:58.462 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:05:58.462 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:05:59.266 INFO:tasks.rados.rados.0.smithi017.stdout:826: done (0 left) 2023-03-29T14:05:59.266 INFO:tasks.rados.rados.0.smithi017.stdout:827: read oid 6 snap -1 2023-03-29T14:05:59.267 INFO:tasks.rados.rados.0.smithi017.stdout:827: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:05:59.267 INFO:tasks.rados.rados.0.smithi017.stdout:828: delete oid 47 current snap is 95 2023-03-29T14:05:59.304 INFO:tasks.rados.rados.0.smithi017.stdout:827: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:05:59.434 INFO:tasks.rados.rados.0.smithi017.stdout:827: done (1 left) 2023-03-29T14:05:59.435 INFO:tasks.rados.rados.0.smithi017.stdout:828: done (0 left) 2023-03-29T14:05:59.435 INFO:tasks.rados.rados.0.smithi017.stdout:829: write oid 5 current snap is 95 2023-03-29T14:05:59.435 INFO:tasks.rados.rados.0.smithi017.stdout:829: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2023-03-29T14:05:59.440 INFO:tasks.rados.rados.0.smithi017.stdout:829: writing smithi017107884-5 from 406835 to 1049389 tid 1 2023-03-29T14:05:59.446 INFO:tasks.rados.rados.0.smithi017.stdout:829: writing smithi017107884-5 from 1532972 to 2039767 tid 2 2023-03-29T14:05:59.450 INFO:tasks.rados.rados.0.smithi017.stdout:829: writing smithi017107884-5 from 2772312 to 2806835 tid 3 2023-03-29T14:05:59.450 INFO:tasks.rados.rados.0.smithi017.stdout:830: snap_create 2023-03-29T14:05:59.487 INFO:tasks.rados.rados.0.smithi017.stdout:829: finishing write tid 1 to smithi017107884-5 2023-03-29T14:05:59.499 INFO:tasks.rados.rados.0.smithi017.stdout:829: finishing write tid 2 to smithi017107884-5 2023-03-29T14:05:59.501 INFO:tasks.rados.rados.0.smithi017.stdout:829: finishing write tid 3 to smithi017107884-5 2023-03-29T14:05:59.543 INFO:tasks.rados.rados.0.smithi017.stdout:829: finishing write tid 4 to smithi017107884-5 2023-03-29T14:05:59.545 INFO:tasks.rados.rados.0.smithi017.stdout:829: finishing write tid 5 to smithi017107884-5 2023-03-29T14:05:59.545 INFO:tasks.rados.rados.0.smithi017.stdout:829: oid 5 updating version 0 to 135 2023-03-29T14:05:59.545 INFO:tasks.rados.rados.0.smithi017.stdout:829: oid 5 updating version 135 to 136 2023-03-29T14:05:59.545 INFO:tasks.rados.rados.0.smithi017.stdout:829: oid 5 version 136 is already newer than 136 2023-03-29T14:05:59.545 INFO:tasks.rados.rados.0.smithi017.stdout:829: oid 5 version 136 is already newer than 136 2023-03-29T14:05:59.545 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 136 (ObjNum 405 snap 95 seq_num 405) dirty exists 2023-03-29T14:05:59.545 INFO:tasks.rados.rados.0.smithi017.stdout:829: left oid 5 (ObjNum 405 snap 95 seq_num 405) 2023-03-29T14:05:59.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2023-03-29T14:06:00.270 INFO:tasks.rados.rados.0.smithi017.stdout:829: done (1 left) 2023-03-29T14:06:00.270 INFO:tasks.rados.rados.0.smithi017.stdout:830: done (0 left) 2023-03-29T14:06:00.271 INFO:tasks.rados.rados.0.smithi017.stdout:831: read oid 3 snap 95 2023-03-29T14:06:00.271 INFO:tasks.rados.rados.0.smithi017.stdout:831: expect deleted 2023-03-29T14:06:00.271 INFO:tasks.rados.rados.0.smithi017.stdout:832: write oid 49 current snap is 96 2023-03-29T14:06:00.271 INFO:tasks.rados.rados.0.smithi017.stdout:832: seq_num 406 ranges {423642=717803,1875101=766806} 2023-03-29T14:06:00.276 INFO:tasks.rados.rados.0.smithi017.stdout:832: writing smithi017107884-49 from 423642 to 1141445 tid 1 2023-03-29T14:06:00.285 INFO:tasks.rados.rados.0.smithi017.stdout:832: writing smithi017107884-49 from 1875101 to 2641907 tid 2 2023-03-29T14:06:00.285 INFO:tasks.rados.rados.0.smithi017.stdout:833: read oid 17 snap -1 2023-03-29T14:06:00.286 INFO:tasks.rados.rados.0.smithi017.stdout:833: expect (ObjNum 378 snap 92 seq_num 378) 2023-03-29T14:06:00.286 INFO:tasks.rados.rados.0.smithi017.stdout:834: snap_create 2023-03-29T14:06:00.307 INFO:tasks.rados.rados.0.smithi017.stdout:833: expect (ObjNum 378 snap 92 seq_num 378) 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: finishing write tid 1 to smithi017107884-49 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: finishing write tid 2 to smithi017107884-49 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: finishing write tid 3 to smithi017107884-49 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: finishing write tid 4 to smithi017107884-49 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: oid 49 updating version 0 to 178 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: oid 49 updating version 178 to 179 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: oid 49 version 179 is already newer than 179 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 179 (ObjNum 406 snap 96 seq_num 406) dirty exists 2023-03-29T14:06:00.454 INFO:tasks.rados.rados.0.smithi017.stdout:832: left oid 49 (ObjNum 406 snap 96 seq_num 406) 2023-03-29T14:06:00.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:00.592 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:00.590+0000 7ff13669d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:00.595 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:00.594+0000 7ff13669d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:00.661 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:00.664 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:00.664 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:01.272 INFO:tasks.rados.rados.0.smithi017.stdout:831: done (3 left) 2023-03-29T14:06:01.272 INFO:tasks.rados.rados.0.smithi017.stdout:832: done (2 left) 2023-03-29T14:06:01.272 INFO:tasks.rados.rados.0.smithi017.stdout:833: done (1 left) 2023-03-29T14:06:01.272 INFO:tasks.rados.rados.0.smithi017.stdout:834: done (0 left) 2023-03-29T14:06:01.272 INFO:tasks.rados.rados.0.smithi017.stdout:835: read oid 1 snap -1 2023-03-29T14:06:01.272 INFO:tasks.rados.rados.0.smithi017.stdout:835: expect (ObjNum 343 snap 82 seq_num 343) 2023-03-29T14:06:01.272 INFO:tasks.rados.rados.0.smithi017.stdout:836: read oid 43 snap -1 2023-03-29T14:06:01.273 INFO:tasks.rados.rados.0.smithi017.stdout:836: expect (ObjNum 346 snap 82 seq_num 346) 2023-03-29T14:06:01.273 INFO:tasks.rados.rados.0.smithi017.stdout:837: read oid 41 snap -1 2023-03-29T14:06:01.273 INFO:tasks.rados.rados.0.smithi017.stdout:837: expect deleted 2023-03-29T14:06:01.273 INFO:tasks.rados.rados.0.smithi017.stdout:838: write (excl) oid 25 current snap is 97 2023-03-29T14:06:01.273 INFO:tasks.rados.rados.0.smithi017.stdout:838: seq_num 407 ranges {440449=793052,1933583=510464} 2023-03-29T14:06:01.279 INFO:tasks.rados.rados.0.smithi017.stdout:838: writing smithi017107884-25 from 440449 to 1233501 tid 1 2023-03-29T14:06:01.286 INFO:tasks.rados.rados.0.smithi017.stdout:838: writing smithi017107884-25 from 1933583 to 2444047 tid 2 2023-03-29T14:06:01.286 INFO:tasks.rados.rados.0.smithi017.stdout:839: delete oid 13 current snap is 97 2023-03-29T14:06:01.380 INFO:tasks.rados.rados.0.smithi017.stdout:835: expect (ObjNum 343 snap 82 seq_num 343) 2023-03-29T14:06:01.568 INFO:tasks.rados.rados.0.smithi017.stdout:838: finishing write tid 1 to smithi017107884-25 2023-03-29T14:06:01.568 INFO:tasks.rados.rados.0.smithi017.stdout:836: expect (ObjNum 346 snap 82 seq_num 346) 2023-03-29T14:06:01.776 INFO:tasks.rados.rados.0.smithi017.stdout:838: finishing write tid 2 to smithi017107884-25 2023-03-29T14:06:01.777 INFO:tasks.rados.rados.0.smithi017.stdout:838: finishing write tid 3 to smithi017107884-25 2023-03-29T14:06:01.777 INFO:tasks.rados.rados.0.smithi017.stdout:838: finishing write tid 4 to smithi017107884-25 2023-03-29T14:06:01.777 INFO:tasks.rados.rados.0.smithi017.stdout:838: oid 25 updating version 0 to 127 2023-03-29T14:06:01.777 INFO:tasks.rados.rados.0.smithi017.stdout:838: oid 25 version 127 is already newer than 127 2023-03-29T14:06:01.777 INFO:tasks.rados.rados.0.smithi017.stdout:838: oid 25 version 127 is already newer than 124 2023-03-29T14:06:01.777 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 127 (ObjNum 407 snap 97 seq_num 407) dirty exists 2023-03-29T14:06:01.777 INFO:tasks.rados.rados.0.smithi017.stdout:838: left oid 25 (ObjNum 407 snap 97 seq_num 407) 2023-03-29T14:06:01.778 INFO:tasks.rados.rados.0.smithi017.stdout:835: done (4 left) 2023-03-29T14:06:01.778 INFO:tasks.rados.rados.0.smithi017.stdout:836: done (3 left) 2023-03-29T14:06:01.778 INFO:tasks.rados.rados.0.smithi017.stdout:837: done (2 left) 2023-03-29T14:06:01.778 INFO:tasks.rados.rados.0.smithi017.stdout:838: done (1 left) 2023-03-29T14:06:01.778 INFO:tasks.rados.rados.0.smithi017.stdout:839: done (0 left) 2023-03-29T14:06:01.778 INFO:tasks.rados.rados.0.smithi017.stdout:840: read oid 22 snap 95 2023-03-29T14:06:01.778 INFO:tasks.rados.rados.0.smithi017.stdout:840: expect (ObjNum 399 snap 94 seq_num 399) 2023-03-29T14:06:01.779 INFO:tasks.rados.rados.0.smithi017.stdout:841: read oid 6 snap -1 2023-03-29T14:06:01.779 INFO:tasks.rados.rados.0.smithi017.stdout:841: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:06:01.779 INFO:tasks.rados.rados.0.smithi017.stdout:842: read oid 24 snap 96 2023-03-29T14:06:01.779 INFO:tasks.rados.rados.0.smithi017.stdout:842: expect (ObjNum 296 snap 78 seq_num 296) 2023-03-29T14:06:01.779 INFO:tasks.rados.rados.0.smithi017.stdout:843: delete oid 28 current snap is 97 2023-03-29T14:06:01.795 INFO:tasks.rados.rados.0.smithi017.stdout:840: expect (ObjNum 399 snap 94 seq_num 399) 2023-03-29T14:06:01.978 INFO:tasks.rados.rados.0.smithi017.stdout:841: expect (ObjNum 315 snap 79 seq_num 315) 2023-03-29T14:06:02.208 INFO:tasks.rados.rados.0.smithi017.stdout:842: expect (ObjNum 296 snap 78 seq_num 296) 2023-03-29T14:06:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:840: done (3 left) 2023-03-29T14:06:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:841: done (2 left) 2023-03-29T14:06:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:842: done (1 left) 2023-03-29T14:06:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:843: done (0 left) 2023-03-29T14:06:02.524 INFO:tasks.rados.rados.0.smithi017.stdout:844: snap_remove snap 95 2023-03-29T14:06:02.550 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:02.550 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:02.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:02.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:02.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:02.679+0000 7f8e89cbc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:02.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:02.685+0000 7f8e89cbc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:02.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:02.795+0000 7f5ecfc6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:02.800 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:02.799+0000 7f5ecfc6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:02.867 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:02.867 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:02.893 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:02.906 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-03-29T14:06:02.140171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":172716278,"num_objects":480,"num_object_clones":384,"num_object_copies":1440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":1992,"num_read_kb":1089801,"num_write":2696,"num_write_kb":705594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1410,"ondisk_log_size":1410,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":983776,"kb_used_data":929088,"kb_used_omap":0,"kb_used_meta":54656,"kb_avail":376503584,"statfs":{"total":386547056640,"available":385539670016,"internally_reserved":0,"allocated":951386112,"data_stored":1142548871,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":55967744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20907908,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":63,"num_read_kb":30430,"num_write":58,"num_write_kb":17034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000673"},"pg_stats":[{"pgid":"3.f","version":"212'56","reported_seq":69,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:00.280409+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:00.280409+0000","last_peered":"2023-03-29T14:06:00.280409+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:00.280409+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10622288,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":4,"num_write":32,"num_write_kb":8458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"209'21","reported_seq":217,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:58.321082+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:05:58.321082+0000","last_peered":"2023-03-29T14:05:58.321082+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:05:58.321082+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5075846,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":17991,"num_write":44,"num_write_kb":12992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"205'112","reported_seq":428,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-03-29T14:06:00.306692+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:00.306692+0000","last_peered":"2023-03-29T14:06:00.306692+0000","last_clean":"2023-03-29T14:06:00.306692+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:00.306692+0000","last_undegraded":"2023-03-29T14:06:00.306692+0000","last_fullsized":"2023-03-29T14:06:00.306692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9210500,"num_objects":38,"num_object_clones":28,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":161,"num_read_kb":113920,"num_write":218,"num_write_kb":64190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"216'127","reported_seq":460,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-03-29T14:06:01.436584+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:01.436584+0000","last_peered":"2023-03-29T14:06:01.436584+0000","last_clean":"2023-03-29T14:06:01.436584+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:01.436584+0000","last_undegraded":"2023-03-29T14:06:01.436584+0000","last_fullsized":"2023-03-29T14:06:01.436584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18084662,"num_objects":56,"num_object_clones":48,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":194,"num_read_kb":118715,"num_write":248,"num_write_kb":69364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"198'101","reported_seq":396,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-03-29T14:05:58.306422+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:05:58.306422+0000","last_peered":"2023-03-29T14:05:58.306422+0000","last_clean":"2023-03-29T14:05:58.306422+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:05:58.306422+0000","last_undegraded":"2023-03-29T14:05:58.306422+0000","last_fullsized":"2023-03-29T14:05:58.306422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18924880,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":85832,"num_write":204,"num_write_kb":61526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":176,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:06:00.288798+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:00.288798+0000","last_peered":"2023-03-29T14:06:00.288798+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:00.288798+0000","last_undegraded":"2023-03-29T14:06:00.288798+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"213'86","reported_seq":359,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:00.284781+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:00.284781+0000","last_peered":"2023-03-29T14:06:00.284781+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:00.284781+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7596830,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":108,"num_read_kb":51657,"num_write":154,"num_write_kb":45076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"209'57","reported_seq":302,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-03-29T14:06:00.276594+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:00.276594+0000","last_peered":"2023-03-29T14:06:00.276594+0000","last_clean":"2023-03-29T14:06:00.276594+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:00.276594+0000","last_undegraded":"2023-03-29T14:06:00.276594+0000","last_fullsized":"2023-03-29T14:06:00.276594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6117748,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":84,"num_read_kb":49141,"num_write":116,"num_write_kb":32686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:05:58.279392+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:05:58.279392+0000","last_peered":"2023-03-29T14:05:58.279392+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:05:58.279392+0000","last_undegraded":"2023-03-29T14:05:58.279392+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":41,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:05:58.284625+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:05:58.284625+0000","last_peered":"2023-03-29T14:05:58.284625+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:05:58.284625+0000","last_undegraded":"2023-03-29T14:05:58.284625+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":179,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:06:00.286576+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:00.286576+0000","last_peered":"2023-03-29T14:06:00.286576+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:00.286576+0000","last_undegraded":"2023-03-29T14:06:00.286576+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"208'62","reported_seq":314,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:58.315701+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:05:58.315701+0000","last_peered":"2023-03-29T14:05:58.315701+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:05:58.315701+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10958362,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":64093,"num_write":128,"num_write_kb":36142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:05:58.329346+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:05:58.329346+0000","last_peered":"2023-03-29T14:05:58.329346+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:05:58.329346+0000","last_undegraded":"2023-03-29T14:05:58.329346+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"212'82","reported_seq":351,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:58.316902+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:05:58.316902+0000","last_peered":"2023-03-29T14:05:58.316902+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:05:58.316902+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11630642,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":45592,"num_write":170,"num_write_kb":51308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":179,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:06:00.274658+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:00.274658+0000","last_peered":"2023-03-29T14:06:00.274658+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:00.274658+0000","last_undegraded":"2023-03-29T14:06:00.274658+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":397,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:58.308687+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:05:58.308687+0000","last_peered":"2023-03-29T14:05:58.308687+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:05:58.308687+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"213'80","reported_seq":390,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:00.290541+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:00.290541+0000","last_peered":"2023-03-29T14:06:00.290541+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:00.290541+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15597226,"num_objects":32,"num_object_clones":24,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":188,"num_read_kb":120474,"num_write":158,"num_write_kb":44220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":32}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":179,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:06:00.291512+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:00.291512+0000","last_peered":"2023-03-29T14:06:00.291512+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:00.291512+0000","last_undegraded":"2023-03-29T14:06:00.291512+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"215'179","reported_seq":582,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:00.394566+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:00.394566+0000","last_peered":"2023-03-29T14:06:00.394566+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:00.394566+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15462836,"num_objects":66,"num_object_clones":52,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":292,"num_read_kb":179124,"num_write":334,"num_write_kb":95938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":66}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":172,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:05:58.314912+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:05:58.314912+0000","last_peered":"2023-03-29T14:05:58.314912+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:05:58.314912+0000","last_undegraded":"2023-03-29T14:05:58.314912+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"202'63","reported_seq":314,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-03-29T14:05:58.323565+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:05:58.323565+0000","last_peered":"2023-03-29T14:05:58.323565+0000","last_clean":"2023-03-29T14:05:58.323565+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:05:58.323565+0000","last_undegraded":"2023-03-29T14:05:58.323565+0000","last_fullsized":"2023-03-29T14:05:58.323565+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8639062,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":55142,"num_write":134,"num_write_kb":37726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"214'136","reported_seq":461,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-03-29T14:05:59.493520+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:05:59.493520+0000","last_peered":"2023-03-29T14:05:59.493520+0000","last_clean":"2023-03-29T14:05:59.493520+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:05:59.493520+0000","last_undegraded":"2023-03-29T14:05:59.493520+0000","last_fullsized":"2023-03-29T14:05:59.493520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723130,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":173,"num_read_kb":98003,"num_write":260,"num_write_kb":76742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:05:58.325917+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:05:58.325917+0000","last_peered":"2023-03-29T14:05:58.325917+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:05:58.325917+0000","last_undegraded":"2023-03-29T14:05:58.325917+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":216,"state":"active+undersized","last_fresh":"2023-03-29T14:06:00.287303+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:00.287303+0000","last_peered":"2023-03-29T14:06:00.287303+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:00.287303+0000","last_undegraded":"2023-03-29T14:06:00.287303+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"212'109","reported_seq":415,"reported_epoch":216,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:05:58.323966+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:05:58.323966+0000","last_peered":"2023-03-29T14:05:58.323966+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:05:58.323966+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18454350,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":156,"num_read_kb":89992,"num_write":214,"num_write_kb":63462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":171535542,"num_objects":476,"num_object_clones":384,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":1850,"num_read_kb":1089681,"num_write":2422,"num_write_kb":701806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1313,"ondisk_log_size":1313,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":261580,"kb_used_data":247940,"kb_used_omap":0,"kb_used_meta":13632,"kb_avail":94110260,"statfs":{"total":96636764160,"available":96368906240,"internally_reserved":0,"allocated":253890560,"data_stored":301304229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13959168},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":293960,"kb_used_data":279296,"kb_used_omap":0,"kb_used_meta":14656,"kb_avail":94077880,"statfs":{"total":96636764160,"available":96335749120,"internally_reserved":0,"allocated":285999104,"data_stored":346499178,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15007744},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":272292,"kb_used_data":258012,"kb_used_omap":0,"kb_used_meta":14272,"kb_avail":94099548,"statfs":{"total":96636764160,"available":96357937152,"internally_reserved":0,"allocated":264204288,"data_stored":321513239,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14614528},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:02.911 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.0 1.0 3.1 2023-03-29T14:06:03.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:03.074+0000 7fc5ed151700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:03.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:03.084+0000 7fc5ed151700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:03.281 INFO:tasks.rados.rados.0.smithi017.stdout:844: done (0 left) 2023-03-29T14:06:03.281 INFO:tasks.rados.rados.0.smithi017.stdout:845: write oid 45 current snap is 97 2023-03-29T14:06:03.281 INFO:tasks.rados.rados.0.smithi017.stdout:845: seq_num 410 ranges {490870=618799,1592676=774144,2793074=97796} 2023-03-29T14:06:03.291 INFO:tasks.rados.rados.0.smithi017.stdout:845: writing smithi017107884-45 from 490870 to 1109669 tid 1 2023-03-29T14:06:03.297 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1] on osd.0 to force-recovery; instructing pg(s) [1.0,3.0] on osd.1 to force-recovery; instructing pg(s) [3.f] on osd.3 to force-recovery; 2023-03-29T14:06:03.298 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:06:03.299 INFO:tasks.rados.rados.0.smithi017.stdout:845: writing smithi017107884-45 from 1592676 to 2366820 tid 2 2023-03-29T14:06:03.303 INFO:tasks.rados.rados.0.smithi017.stdout:845: writing smithi017107884-45 from 2793074 to 2890870 tid 3 2023-03-29T14:06:03.303 INFO:tasks.rados.rados.0.smithi017.stdout:846: write oid 46 current snap is 97 2023-03-29T14:06:03.303 INFO:tasks.rados.rados.0.smithi017.stdout:846: seq_num 411 ranges {507677=410401,1367511=517802,2420497=487180} 2023-03-29T14:06:03.310 INFO:tasks.rados.rados.0.smithi017.stdout:846: writing smithi017107884-46 from 507677 to 918078 tid 1 2023-03-29T14:06:03.315 INFO:tasks.rados.rados.0.smithi017.stdout:846: writing smithi017107884-46 from 1367511 to 1885313 tid 2 2023-03-29T14:06:03.321 INFO:tasks.rados.rados.0.smithi017.stdout:846: writing smithi017107884-46 from 2420497 to 2907677 tid 3 2023-03-29T14:06:03.321 INFO:tasks.rados.rados.0.smithi017.stdout:847: read oid 13 snap 96 2023-03-29T14:06:03.321 INFO:tasks.rados.rados.0.smithi017.stdout:847: expect deleted 2023-03-29T14:06:03.321 INFO:tasks.rados.rados.0.smithi017.stdout:848: write (excl) oid 1 current snap is 97 2023-03-29T14:06:03.321 INFO:tasks.rados.rados.0.smithi017.stdout:848: seq_num 412 ranges {524484=485650,1425993=661460,2847920=76564} 2023-03-29T14:06:03.326 INFO:tasks.rados.rados.0.smithi017.stdout:848: writing smithi017107884-1 from 524484 to 1010134 tid 1 2023-03-29T14:06:03.333 INFO:tasks.rados.rados.0.smithi017.stdout:848: writing smithi017107884-1 from 1425993 to 2087453 tid 2 2023-03-29T14:06:03.337 INFO:tasks.rados.rados.0.smithi017.stdout:848: writing smithi017107884-1 from 2847920 to 2924484 tid 3 2023-03-29T14:06:03.337 INFO:tasks.rados.rados.0.smithi017.stdout:849: read oid 19 snap 96 2023-03-29T14:06:03.337 INFO:tasks.rados.rados.0.smithi017.stdout:849: expect (ObjNum 394 snap 94 seq_num 394) 2023-03-29T14:06:03.337 INFO:tasks.rados.rados.0.smithi017.stdout:850: write (excl) oid 42 current snap is 97 2023-03-29T14:06:03.338 INFO:tasks.rados.rados.0.smithi017.stdout:850: seq_num 413 ranges {541291=560899,1884475=521471} 2023-03-29T14:06:03.343 INFO:tasks.rados.rados.0.smithi017.stdout:850: writing smithi017107884-42 from 541291 to 1102190 tid 1 2023-03-29T14:06:03.350 INFO:tasks.rados.rados.0.smithi017.stdout:850: writing smithi017107884-42 from 1884475 to 2405946 tid 2 2023-03-29T14:06:03.351 INFO:tasks.rados.rados.0.smithi017.stdout:851: delete oid 29 current snap is 97 2023-03-29T14:06:03.444 INFO:tasks.rados.rados.0.smithi017.stdout:848: finishing write tid 1 to smithi017107884-1 2023-03-29T14:06:03.454 INFO:tasks.rados.rados.0.smithi017.stdout:850: finishing write tid 1 to smithi017107884-42 2023-03-29T14:06:03.466 INFO:tasks.rados.rados.0.smithi017.stdout:845: finishing write tid 1 to smithi017107884-45 2023-03-29T14:06:03.477 INFO:tasks.rados.rados.0.smithi017.stdout:845: finishing write tid 2 to smithi017107884-45 2023-03-29T14:06:03.481 INFO:tasks.rados.rados.0.smithi017.stdout:845: finishing write tid 3 to smithi017107884-45 2023-03-29T14:06:03.486 INFO:tasks.rados.rados.0.smithi017.stdout:851: done (6 left) 2023-03-29T14:06:03.486 INFO:tasks.rados.rados.0.smithi017.stdout:852: delete oid 36 current snap is 97 2023-03-29T14:06:03.517 INFO:tasks.rados.rados.0.smithi017.stdout:850: finishing write tid 2 to smithi017107884-42 2023-03-29T14:06:03.545 INFO:tasks.rados.rados.0.smithi017.stdout:848: finishing write tid 2 to smithi017107884-1 2023-03-29T14:06:03.547 INFO:tasks.rados.rados.0.smithi017.stdout:848: finishing write tid 3 to smithi017107884-1 2023-03-29T14:06:03.548 INFO:tasks.rados.rados.0.smithi017.stdout:848: finishing write tid 4 to smithi017107884-1 2023-03-29T14:06:03.552 INFO:tasks.rados.rados.0.smithi017.stdout:848: finishing write tid 5 to smithi017107884-1 2023-03-29T14:06:03.552 INFO:tasks.rados.rados.0.smithi017.stdout:848: oid 1 updating version 0 to 110 2023-03-29T14:06:03.552 INFO:tasks.rados.rados.0.smithi017.stdout:848: oid 1 updating version 110 to 114 2023-03-29T14:06:03.552 INFO:tasks.rados.rados.0.smithi017.stdout:848: oid 1 version 114 is already newer than 114 2023-03-29T14:06:03.552 INFO:tasks.rados.rados.0.smithi017.stdout:848: oid 1 version 114 is already newer than 114 2023-03-29T14:06:03.552 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 114 (ObjNum 412 snap 97 seq_num 412) dirty exists 2023-03-29T14:06:03.552 INFO:tasks.rados.rados.0.smithi017.stdout:848: left oid 1 (ObjNum 412 snap 97 seq_num 412) 2023-03-29T14:06:03.554 INFO:tasks.rados.rados.0.smithi017.stdout:850: finishing write tid 3 to smithi017107884-42 2023-03-29T14:06:03.556 INFO:tasks.rados.rados.0.smithi017.stdout:850: finishing write tid 4 to smithi017107884-42 2023-03-29T14:06:03.556 INFO:tasks.rados.rados.0.smithi017.stdout:850: oid 42 updating version 0 to 130 2023-03-29T14:06:03.556 INFO:tasks.rados.rados.0.smithi017.stdout:850: oid 42 version 130 is already newer than 130 2023-03-29T14:06:03.556 INFO:tasks.rados.rados.0.smithi017.stdout:850: oid 42 version 130 is already newer than 128 2023-03-29T14:06:03.556 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 130 (ObjNum 413 snap 97 seq_num 413) dirty exists 2023-03-29T14:06:03.556 INFO:tasks.rados.rados.0.smithi017.stdout:850: left oid 42 (ObjNum 413 snap 97 seq_num 413) 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: finishing write tid 4 to smithi017107884-45 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: finishing write tid 5 to smithi017107884-45 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: oid 45 updating version 0 to 105 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: oid 45 version 105 is already newer than 105 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: oid 45 version 105 is already newer than 105 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: oid 45 version 105 is already newer than 105 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 105 (ObjNum 410 snap 97 seq_num 410) dirty exists 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: left oid 45 (ObjNum 410 snap 97 seq_num 410) 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:845: done (6 left) 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:847: done (5 left) 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:848: done (4 left) 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:850: done (3 left) 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:852: done (2 left) 2023-03-29T14:06:03.569 INFO:tasks.rados.rados.0.smithi017.stdout:853: delete oid 23 current snap is 97 2023-03-29T14:06:03.594 INFO:tasks.rados.rados.0.smithi017.stdout:846: finishing write tid 1 to smithi017107884-46 2023-03-29T14:06:03.602 INFO:tasks.rados.rados.0.smithi017.stdout:849: expect (ObjNum 394 snap 94 seq_num 394) 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: finishing write tid 2 to smithi017107884-46 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: finishing write tid 3 to smithi017107884-46 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: finishing write tid 4 to smithi017107884-46 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: finishing write tid 5 to smithi017107884-46 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: oid 46 updating version 0 to 110 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: oid 46 version 110 is already newer than 110 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: oid 46 version 110 is already newer than 110 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: oid 46 version 110 is already newer than 110 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 110 (ObjNum 411 snap 97 seq_num 411) dirty exists 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: left oid 46 (ObjNum 411 snap 97 seq_num 411) 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:846: done (2 left) 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:849: done (1 left) 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:853: done (0 left) 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:854: read oid 22 snap -1 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:854: expect (ObjNum 399 snap 94 seq_num 399) 2023-03-29T14:06:03.765 INFO:tasks.rados.rados.0.smithi017.stdout:855: snap_remove snap 96 2023-03-29T14:06:04.285 INFO:tasks.rados.rados.0.smithi017.stdout:855: done (1 left) 2023-03-29T14:06:04.286 INFO:tasks.rados.rados.0.smithi017.stdout:856: delete oid 3 current snap is 97 2023-03-29T14:06:04.286 INFO:tasks.rados.rados.0.smithi017.stdout:854: expect (ObjNum 399 snap 94 seq_num 399) 2023-03-29T14:06:04.392 INFO:tasks.rados.rados.0.smithi017.stdout:854: done (1 left) 2023-03-29T14:06:04.392 INFO:tasks.rados.rados.0.smithi017.stdout:856: done (0 left) 2023-03-29T14:06:04.392 INFO:tasks.rados.rados.0.smithi017.stdout:857: delete oid 16 current snap is 97 2023-03-29T14:06:04.437 INFO:tasks.rados.rados.0.smithi017.stdout:857: done (0 left) 2023-03-29T14:06:04.437 INFO:tasks.rados.rados.0.smithi017.stdout:858: write oid 4 current snap is 97 2023-03-29T14:06:04.437 INFO:tasks.rados.rados.0.smithi017.stdout:858: seq_num 419 ranges {642133=728746,2119014=532478} 2023-03-29T14:06:04.444 INFO:tasks.rados.rados.0.smithi017.stdout:858: writing smithi017107884-4 from 642133 to 1370879 tid 1 2023-03-29T14:06:04.452 INFO:tasks.rados.rados.0.smithi017.stdout:858: writing smithi017107884-4 from 2119014 to 2651492 tid 2 2023-03-29T14:06:04.452 INFO:tasks.rados.rados.0.smithi017.stdout:859: read oid 11 snap -1 2023-03-29T14:06:04.452 INFO:tasks.rados.rados.0.smithi017.stdout:859: expect (ObjNum 395 snap 94 seq_num 395) 2023-03-29T14:06:04.452 INFO:tasks.rados.rados.0.smithi017.stdout:860: read oid 44 snap -1 2023-03-29T14:06:04.452 INFO:tasks.rados.rados.0.smithi017.stdout:860: expect (ObjNum 362 snap 89 seq_num 362) 2023-03-29T14:06:04.452 INFO:tasks.rados.rados.0.smithi017.stdout:861: delete oid 40 current snap is 97 2023-03-29T14:06:04.464 INFO:tasks.rados.rados.0.smithi017.stdout:858: finishing write tid 1 to smithi017107884-4 2023-03-29T14:06:04.517 INFO:tasks.rados.rados.0.smithi017.stdout:859: expect (ObjNum 395 snap 94 seq_num 395) 2023-03-29T14:06:04.622 INFO:tasks.rados.rados.0.smithi017.stdout:859: done (3 left) 2023-03-29T14:06:04.622 INFO:tasks.rados.rados.0.smithi017.stdout:861: done (2 left) 2023-03-29T14:06:04.622 INFO:tasks.rados.rados.0.smithi017.stdout:862: read oid 45 snap -1 2023-03-29T14:06:04.622 INFO:tasks.rados.rados.0.smithi017.stdout:862: expect (ObjNum 410 snap 97 seq_num 410) 2023-03-29T14:06:04.622 INFO:tasks.rados.rados.0.smithi017.stdout:863: write (excl) oid 29 current snap is 97 2023-03-29T14:06:04.622 INFO:tasks.rados.rados.0.smithi017.stdout:858: finishing write tid 2 to smithi017107884-4 2023-03-29T14:06:04.622 INFO:tasks.rados.rados.0.smithi017.stdout:863: seq_num 421 ranges {675747=479244,1835978=536147} 2023-03-29T14:06:04.628 INFO:tasks.rados.rados.0.smithi017.stdout:863: writing smithi017107884-29 from 675747 to 1154991 tid 1 2023-03-29T14:06:04.635 INFO:tasks.rados.rados.0.smithi017.stdout:863: writing smithi017107884-29 from 1835978 to 2372125 tid 2 2023-03-29T14:06:04.635 INFO:tasks.rados.rados.0.smithi017.stdout:864: read oid 37 snap -1 2023-03-29T14:06:04.635 INFO:tasks.rados.rados.0.smithi017.stdout:864: expect (ObjNum 300 snap 78 seq_num 300) 2023-03-29T14:06:04.635 INFO:tasks.rados.rados.0.smithi017.stdout:865: snap_create 2023-03-29T14:06:04.635 INFO:tasks.rados.rados.0.smithi017.stdout:860: expect (ObjNum 362 snap 89 seq_num 362) 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:858: finishing write tid 3 to smithi017107884-4 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:858: finishing write tid 4 to smithi017107884-4 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:858: oid 4 updating version 0 to 117 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:858: oid 4 version 117 is already newer than 117 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:858: oid 4 version 117 is already newer than 117 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 117 (ObjNum 419 snap 97 seq_num 419) dirty exists 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:858: left oid 4 (ObjNum 419 snap 97 seq_num 419) 2023-03-29T14:06:04.749 INFO:tasks.rados.rados.0.smithi017.stdout:862: expect (ObjNum 410 snap 97 seq_num 410) 2023-03-29T14:06:04.860 INFO:tasks.rados.rados.0.smithi017.stdout:863: finishing write tid 1 to smithi017107884-29 2023-03-29T14:06:04.860 INFO:tasks.rados.rados.0.smithi017.stdout:863: finishing write tid 2 to smithi017107884-29 2023-03-29T14:06:04.860 INFO:tasks.rados.rados.0.smithi017.stdout:864: expect (ObjNum 300 snap 78 seq_num 300) 2023-03-29T14:06:04.868 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:04.979 INFO:tasks.rados.rados.0.smithi017.stdout:863: finishing write tid 3 to smithi017107884-29 2023-03-29T14:06:04.979 INFO:tasks.rados.rados.0.smithi017.stdout:863: finishing write tid 4 to smithi017107884-29 2023-03-29T14:06:04.979 INFO:tasks.rados.rados.0.smithi017.stdout:863: oid 29 updating version 0 to 67 2023-03-29T14:06:04.979 INFO:tasks.rados.rados.0.smithi017.stdout:863: oid 29 version 67 is already newer than 67 2023-03-29T14:06:04.980 INFO:tasks.rados.rados.0.smithi017.stdout:863: oid 29 version 67 is already newer than 65 2023-03-29T14:06:04.980 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 67 (ObjNum 421 snap 97 seq_num 421) dirty exists 2023-03-29T14:06:04.980 INFO:tasks.rados.rados.0.smithi017.stdout:863: left oid 29 (ObjNum 421 snap 97 seq_num 421) 2023-03-29T14:06:04.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:04.995+0000 7f7eef438700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:04.999+0000 7f7eef438700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:05.062 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:05.065 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:05.065 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:05.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~55s 2023-03-29T14:06:05.288 INFO:tasks.rados.rados.0.smithi017.stdout:858: done (5 left) 2023-03-29T14:06:05.288 INFO:tasks.rados.rados.0.smithi017.stdout:860: done (4 left) 2023-03-29T14:06:05.288 INFO:tasks.rados.rados.0.smithi017.stdout:862: done (3 left) 2023-03-29T14:06:05.288 INFO:tasks.rados.rados.0.smithi017.stdout:863: done (2 left) 2023-03-29T14:06:05.288 INFO:tasks.rados.rados.0.smithi017.stdout:864: done (1 left) 2023-03-29T14:06:05.288 INFO:tasks.rados.rados.0.smithi017.stdout:865: done (0 left) 2023-03-29T14:06:05.289 INFO:tasks.rados.rados.0.smithi017.stdout:866: delete oid 17 current snap is 98 2023-03-29T14:06:05.386 INFO:tasks.rados.rados.0.smithi017.stdout:866: done (0 left) 2023-03-29T14:06:05.386 INFO:tasks.rados.rados.0.smithi017.stdout:867: read oid 49 snap -1 2023-03-29T14:06:05.386 INFO:tasks.rados.rados.0.smithi017.stdout:867: expect (ObjNum 406 snap 96 seq_num 406) 2023-03-29T14:06:05.386 INFO:tasks.rados.rados.0.smithi017.stdout:868: delete oid 28 current snap is 98 2023-03-29T14:06:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:868: done (1 left) 2023-03-29T14:06:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:869: read oid 13 snap -1 2023-03-29T14:06:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:869: expect deleted 2023-03-29T14:06:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:870: write oid 34 current snap is 98 2023-03-29T14:06:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:870: seq_num 424 ranges {726168=704991,1895071=683474} 2023-03-29T14:06:05.396 INFO:tasks.rados.rados.0.smithi017.stdout:870: writing smithi017107884-34 from 726168 to 1431159 tid 1 2023-03-29T14:06:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:870: writing smithi017107884-34 from 1895071 to 2578545 tid 2 2023-03-29T14:06:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:871: write (excl) oid 9 current snap is 98 2023-03-29T14:06:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:871: seq_num 425 ranges {742975=780240,1953553=427132,2971713=171262} 2023-03-29T14:06:05.413 INFO:tasks.rados.rados.0.smithi017.stdout:871: writing smithi017107884-9 from 742975 to 1523215 tid 1 2023-03-29T14:06:05.420 INFO:tasks.rados.rados.0.smithi017.stdout:871: writing smithi017107884-9 from 1953553 to 2380685 tid 2 2023-03-29T14:06:05.422 INFO:tasks.rados.rados.0.smithi017.stdout:871: writing smithi017107884-9 from 2971713 to 3142975 tid 3 2023-03-29T14:06:05.423 INFO:tasks.rados.rados.0.smithi017.stdout:872: write oid 50 current snap is 98 2023-03-29T14:06:05.423 INFO:tasks.rados.rados.0.smithi017.stdout:872: seq_num 426 ranges {759782=571842,2012035=687143} 2023-03-29T14:06:05.430 INFO:tasks.rados.rados.0.smithi017.stdout:872: writing smithi017107884-50 from 759782 to 1331624 tid 1 2023-03-29T14:06:05.437 INFO:tasks.rados.rados.0.smithi017.stdout:872: writing smithi017107884-50 from 2012035 to 2699178 tid 2 2023-03-29T14:06:05.438 INFO:tasks.rados.rados.0.smithi017.stdout:873: write (excl) oid 20 current snap is 98 2023-03-29T14:06:05.438 INFO:tasks.rados.rados.0.smithi017.stdout:873: seq_num 427 ranges {776589=647091,2070517=430801,3026559=150030} 2023-03-29T14:06:05.445 INFO:tasks.rados.rados.0.smithi017.stdout:873: writing smithi017107884-20 from 776589 to 1423680 tid 1 2023-03-29T14:06:05.451 INFO:tasks.rados.rados.0.smithi017.stdout:873: writing smithi017107884-20 from 2070517 to 2501318 tid 2 2023-03-29T14:06:05.455 INFO:tasks.rados.rados.0.smithi017.stdout:873: writing smithi017107884-20 from 3026559 to 3176589 tid 3 2023-03-29T14:06:05.455 INFO:tasks.rados.rados.0.smithi017.stdout:874: write (excl) oid 42 current snap is 98 2023-03-29T14:06:05.455 INFO:tasks.rados.rados.0.smithi017.stdout:867: expect (ObjNum 406 snap 96 seq_num 406) 2023-03-29T14:06:05.568 INFO:tasks.rados.rados.0.smithi017.stdout:874: seq_num 428 ranges {793396=722340,2128999=690812} 2023-03-29T14:06:05.577 INFO:tasks.rados.rados.0.smithi017.stdout:874: writing smithi017107884-42 from 793396 to 1515736 tid 1 2023-03-29T14:06:05.584 INFO:tasks.rados.rados.0.smithi017.stdout:874: writing smithi017107884-42 from 2128999 to 2819811 tid 2 2023-03-29T14:06:05.585 INFO:tasks.rados.rados.0.smithi017.stdout:867: done (6 left) 2023-03-29T14:06:05.585 INFO:tasks.rados.rados.0.smithi017.stdout:875: write oid 29 current snap is 98 2023-03-29T14:06:05.585 INFO:tasks.rados.rados.0.smithi017.stdout:875: seq_num 429 ranges {410203=797589,1787481=434470,2681405=528798} 2023-03-29T14:06:05.591 INFO:tasks.rados.rados.0.smithi017.stdout:875: writing smithi017107884-29 from 410203 to 1207792 tid 1 2023-03-29T14:06:05.596 INFO:tasks.rados.rados.0.smithi017.stdout:875: writing smithi017107884-29 from 1787481 to 2221951 tid 2 2023-03-29T14:06:05.602 INFO:tasks.rados.rados.0.smithi017.stdout:875: writing smithi017107884-29 from 2681405 to 3210203 tid 3 2023-03-29T14:06:05.602 INFO:tasks.rados.rados.0.smithi017.stdout:876: read oid 19 snap -1 2023-03-29T14:06:05.602 INFO:tasks.rados.rados.0.smithi017.stdout:876: expect (ObjNum 394 snap 94 seq_num 394) 2023-03-29T14:06:05.602 INFO:tasks.rados.rados.0.smithi017.stdout:877: write (excl) oid 49 current snap is 98 2023-03-29T14:06:05.603 INFO:tasks.rados.rados.0.smithi017.stdout:871: finishing write tid 1 to smithi017107884-9 2023-03-29T14:06:05.603 INFO:tasks.rados.rados.0.smithi017.stdout:877: seq_num 430 ranges {427010=472838,1329610=694481,2708828=518182} 2023-03-29T14:06:05.608 INFO:tasks.rados.rados.0.smithi017.stdout:877: writing smithi017107884-49 from 427010 to 899848 tid 1 2023-03-29T14:06:05.614 INFO:tasks.rados.rados.0.smithi017.stdout:877: writing smithi017107884-49 from 1329610 to 2024091 tid 2 2023-03-29T14:06:05.621 INFO:tasks.rados.rados.0.smithi017.stdout:877: writing smithi017107884-49 from 2708828 to 3227010 tid 3 2023-03-29T14:06:05.621 INFO:tasks.rados.rados.0.smithi017.stdout:878: snap_create 2023-03-29T14:06:05.621 INFO:tasks.rados.rados.0.smithi017.stdout:870: finishing write tid 1 to smithi017107884-34 2023-03-29T14:06:05.621 INFO:tasks.rados.rados.0.smithi017.stdout:871: finishing write tid 2 to smithi017107884-9 2023-03-29T14:06:05.628 INFO:tasks.rados.rados.0.smithi017.stdout:870: finishing write tid 2 to smithi017107884-34 2023-03-29T14:06:05.629 INFO:tasks.rados.rados.0.smithi017.stdout:871: finishing write tid 3 to smithi017107884-9 2023-03-29T14:06:05.640 INFO:tasks.rados.rados.0.smithi017.stdout:872: finishing write tid 1 to smithi017107884-50 2023-03-29T14:06:05.642 INFO:tasks.rados.rados.0.smithi017.stdout:870: finishing write tid 3 to smithi017107884-34 2023-03-29T14:06:05.653 INFO:tasks.rados.rados.0.smithi017.stdout:870: finishing write tid 4 to smithi017107884-34 2023-03-29T14:06:05.653 INFO:tasks.rados.rados.0.smithi017.stdout:870: oid 34 updating version 0 to 114 2023-03-29T14:06:05.653 INFO:tasks.rados.rados.0.smithi017.stdout:870: oid 34 version 114 is already newer than 114 2023-03-29T14:06:05.653 INFO:tasks.rados.rados.0.smithi017.stdout:870: oid 34 version 114 is already newer than 114 2023-03-29T14:06:05.653 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 114 (ObjNum 424 snap 98 seq_num 424) dirty exists 2023-03-29T14:06:05.653 INFO:tasks.rados.rados.0.smithi017.stdout:870: left oid 34 (ObjNum 424 snap 98 seq_num 424) 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:871: finishing write tid 4 to smithi017107884-9 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:871: finishing write tid 5 to smithi017107884-9 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:871: oid 9 updating version 0 to 132 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:871: oid 9 updating version 132 to 136 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:871: oid 9 version 136 is already newer than 136 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:871: oid 9 version 136 is already newer than 136 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 136 (ObjNum 425 snap 98 seq_num 425) dirty exists 2023-03-29T14:06:05.670 INFO:tasks.rados.rados.0.smithi017.stdout:871: left oid 9 (ObjNum 425 snap 98 seq_num 425) 2023-03-29T14:06:05.690 INFO:tasks.rados.rados.0.smithi017.stdout:872: finishing write tid 2 to smithi017107884-50 2023-03-29T14:06:05.720 INFO:tasks.rados.rados.0.smithi017.stdout:874: finishing write tid 1 to smithi017107884-42 2023-03-29T14:06:05.726 INFO:tasks.rados.rados.0.smithi017.stdout:875: finishing write tid 1 to smithi017107884-29 2023-03-29T14:06:05.747 INFO:tasks.rados.rados.0.smithi017.stdout:872: finishing write tid 3 to smithi017107884-50 2023-03-29T14:06:05.756 INFO:tasks.rados.rados.0.smithi017.stdout:872: finishing write tid 4 to smithi017107884-50 2023-03-29T14:06:05.757 INFO:tasks.rados.rados.0.smithi017.stdout:872: oid 50 updating version 0 to 87 2023-03-29T14:06:05.757 INFO:tasks.rados.rados.0.smithi017.stdout:872: oid 50 version 87 is already newer than 87 2023-03-29T14:06:05.757 INFO:tasks.rados.rados.0.smithi017.stdout:872: oid 50 version 87 is already newer than 87 2023-03-29T14:06:05.757 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 87 (ObjNum 426 snap 98 seq_num 426) dirty exists 2023-03-29T14:06:05.757 INFO:tasks.rados.rados.0.smithi017.stdout:872: left oid 50 (ObjNum 426 snap 98 seq_num 426) 2023-03-29T14:06:05.760 INFO:tasks.rados.rados.0.smithi017.stdout:873: finishing write tid 1 to smithi017107884-20 2023-03-29T14:06:05.785 INFO:tasks.rados.rados.0.smithi017.stdout:876: expect (ObjNum 394 snap 94 seq_num 394) 2023-03-29T14:06:05.947 INFO:tasks.rados.rados.0.smithi017.stdout:874: finishing write tid 2 to smithi017107884-42 2023-03-29T14:06:05.947 INFO:tasks.rados.rados.0.smithi017.stdout:875: finishing write tid 2 to smithi017107884-29 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:874: finishing write tid 3 to smithi017107884-42 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:874: finishing write tid 4 to smithi017107884-42 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:874: oid 42 updating version 0 to 140 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:874: oid 42 version 140 is already newer than 140 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:874: oid 42 version 140 is already newer than 137 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 140 (ObjNum 428 snap 98 seq_num 428) dirty exists 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:874: left oid 42 (ObjNum 428 snap 98 seq_num 428) 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:877: finishing write tid 1 to smithi017107884-49 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: finishing write tid 3 to smithi017107884-29 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: finishing write tid 4 to smithi017107884-29 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: finishing write tid 5 to smithi017107884-29 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: oid 29 updating version 0 to 72 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: oid 29 version 72 is already newer than 72 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: oid 29 version 72 is already newer than 72 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: oid 29 version 72 is already newer than 72 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 72 (ObjNum 429 snap 98 seq_num 429) dirty exists 2023-03-29T14:06:05.948 INFO:tasks.rados.rados.0.smithi017.stdout:875: left oid 29 (ObjNum 429 snap 98 seq_num 429) 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: finishing write tid 2 to smithi017107884-49 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: finishing write tid 3 to smithi017107884-49 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: finishing write tid 2 to smithi017107884-20 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: finishing write tid 4 to smithi017107884-49 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: finishing write tid 5 to smithi017107884-49 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: oid 49 updating version 0 to 182 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: oid 49 updating version 182 to 186 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: oid 49 version 186 is already newer than 186 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: oid 49 version 186 is already newer than 186 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 186 (ObjNum 430 snap 98 seq_num 430) dirty exists 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:877: left oid 49 (ObjNum 430 snap 98 seq_num 430) 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: finishing write tid 3 to smithi017107884-20 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: finishing write tid 4 to smithi017107884-20 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: finishing write tid 5 to smithi017107884-20 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: oid 20 updating version 0 to 119 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: oid 20 version 119 is already newer than 115 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: oid 20 version 119 is already newer than 119 2023-03-29T14:06:05.949 INFO:tasks.rados.rados.0.smithi017.stdout:873: oid 20 version 119 is already newer than 119 2023-03-29T14:06:05.950 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 119 (ObjNum 427 snap 98 seq_num 427) dirty exists 2023-03-29T14:06:05.950 INFO:tasks.rados.rados.0.smithi017.stdout:873: left oid 20 (ObjNum 427 snap 98 seq_num 427) 2023-03-29T14:06:06.292 INFO:tasks.rados.rados.0.smithi017.stdout:869: done (9 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:870: done (8 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:871: done (7 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:872: done (6 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:873: done (5 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:874: done (4 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:875: done (3 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:876: done (2 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:877: done (1 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:878: done (0 left) 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:879: write (excl) oid 48 current snap is 99 2023-03-29T14:06:06.293 INFO:tasks.rados.rados.0.smithi017.stdout:879: seq_num 431 ranges {443817=548087,1788092=438139,3019898=223919} 2023-03-29T14:06:06.298 INFO:tasks.rados.rados.0.smithi017.stdout:879: writing smithi017107884-48 from 443817 to 991904 tid 1 2023-03-29T14:06:06.305 INFO:tasks.rados.rados.0.smithi017.stdout:879: writing smithi017107884-48 from 1788092 to 2226231 tid 2 2023-03-29T14:06:06.310 INFO:tasks.rados.rados.0.smithi017.stdout:879: writing smithi017107884-48 from 3019898 to 3243817 tid 3 2023-03-29T14:06:06.310 INFO:tasks.rados.rados.0.smithi017.stdout:880: delete oid 24 current snap is 99 2023-03-29T14:06:06.387 INFO:tasks.rados.rados.0.smithi017.stdout:879: finishing write tid 1 to smithi017107884-48 2023-03-29T14:06:06.388 INFO:tasks.rados.rados.0.smithi017.stdout:880: done (1 left) 2023-03-29T14:06:06.388 INFO:tasks.rados.rados.0.smithi017.stdout:881: snap_remove snap 98 2023-03-29T14:06:07.066 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:07.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:07.194+0000 7f9d1ecf0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:07.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:07.198+0000 7f9d1ecf0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:07.266 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:07.269 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:07.270 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:07.296 INFO:tasks.rados.rados.0.smithi017.stdout:881: done (1 left) 2023-03-29T14:06:07.296 INFO:tasks.rados.rados.0.smithi017.stdout:882: write (excl) oid 42 current snap is 99 2023-03-29T14:06:07.296 INFO:tasks.rados.rados.0.smithi017.stdout:879: finishing write tid 2 to smithi017107884-48 2023-03-29T14:06:07.296 INFO:tasks.rados.rados.0.smithi017.stdout:882: seq_num 433 ranges {477431=698585,1905056=441808,3074744=202687} 2023-03-29T14:06:07.302 INFO:tasks.rados.rados.0.smithi017.stdout:882: writing smithi017107884-42 from 477431 to 1176016 tid 1 2023-03-29T14:06:07.308 INFO:tasks.rados.rados.0.smithi017.stdout:882: writing smithi017107884-42 from 1905056 to 2346864 tid 2 2023-03-29T14:06:07.314 INFO:tasks.rados.rados.0.smithi017.stdout:882: writing smithi017107884-42 from 3074744 to 3277431 tid 3 2023-03-29T14:06:07.314 INFO:tasks.rados.rados.0.smithi017.stdout:883: snap_remove snap 97 2023-03-29T14:06:08.216 INFO:tasks.rados.rados.0.smithi017.stdout:883: done (2 left) 2023-03-29T14:06:08.216 INFO:tasks.rados.rados.0.smithi017.stdout:884: write (excl) oid 22 current snap is 99 2023-03-29T14:06:08.216 INFO:tasks.rados.rados.0.smithi017.stdout:879: finishing write tid 3 to smithi017107884-48 2023-03-29T14:06:08.216 INFO:tasks.rados.rados.0.smithi017.stdout:884: seq_num 434 ranges {494238=490187,1563538=701819,2818520=475718} 2023-03-29T14:06:08.221 INFO:tasks.rados.rados.0.smithi017.stdout:884: writing smithi017107884-22 from 494238 to 984425 tid 1 2023-03-29T14:06:08.228 INFO:tasks.rados.rados.0.smithi017.stdout:884: writing smithi017107884-22 from 1563538 to 2265357 tid 2 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:884: writing smithi017107884-22 from 2818520 to 3294238 tid 3 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:885: delete oid 16 current snap is 99 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:879: finishing write tid 4 to smithi017107884-48 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:879: finishing write tid 5 to smithi017107884-48 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:879: oid 48 updating version 0 to 123 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:879: oid 48 version 123 is already newer than 123 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:879: oid 48 version 123 is already newer than 120 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:879: oid 48 version 123 is already newer than 123 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 123 (ObjNum 431 snap 99 seq_num 431) dirty exists 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:879: left oid 48 (ObjNum 431 snap 99 seq_num 431) 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:882: finishing write tid 1 to smithi017107884-42 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:882: finishing write tid 2 to smithi017107884-42 2023-03-29T14:06:08.235 INFO:tasks.rados.rados.0.smithi017.stdout:882: finishing write tid 3 to smithi017107884-42 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:882: finishing write tid 4 to smithi017107884-42 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:882: finishing write tid 5 to smithi017107884-42 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:882: oid 42 updating version 0 to 144 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:882: oid 42 version 144 is already newer than 144 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:882: oid 42 version 144 is already newer than 144 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:882: oid 42 version 144 is already newer than 141 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 144 (ObjNum 433 snap 99 seq_num 433) dirty exists 2023-03-29T14:06:08.236 INFO:tasks.rados.rados.0.smithi017.stdout:882: left oid 42 (ObjNum 433 snap 99 seq_num 433) 2023-03-29T14:06:08.296 INFO:tasks.rados.rados.0.smithi017.stdout:884: finishing write tid 1 to smithi017107884-22 2023-03-29T14:06:08.314 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:08.315 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:08.315 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:08.349 INFO:tasks.rados.rados.0.smithi017.stdout:884: finishing write tid 2 to smithi017107884-22 2023-03-29T14:06:08.350 INFO:tasks.rados.rados.0.smithi017.stdout:884: finishing write tid 3 to smithi017107884-22 2023-03-29T14:06:08.350 INFO:tasks.rados.rados.0.smithi017.stdout:884: finishing write tid 4 to smithi017107884-22 2023-03-29T14:06:08.352 INFO:tasks.rados.rados.0.smithi017.stdout:884: finishing write tid 5 to smithi017107884-22 2023-03-29T14:06:08.352 INFO:tasks.rados.rados.0.smithi017.stdout:884: oid 22 updating version 0 to 61 2023-03-29T14:06:08.352 INFO:tasks.rados.rados.0.smithi017.stdout:884: oid 22 version 61 is already newer than 61 2023-03-29T14:06:08.352 INFO:tasks.rados.rados.0.smithi017.stdout:884: oid 22 version 61 is already newer than 58 2023-03-29T14:06:08.352 INFO:tasks.rados.rados.0.smithi017.stdout:884: oid 22 version 61 is already newer than 61 2023-03-29T14:06:08.352 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 61 (ObjNum 434 snap 99 seq_num 434) dirty exists 2023-03-29T14:06:08.352 INFO:tasks.rados.rados.0.smithi017.stdout:884: left oid 22 (ObjNum 434 snap 99 seq_num 434) 2023-03-29T14:06:08.353 INFO:tasks.rados.rados.0.smithi017.stdout:879: done (3 left) 2023-03-29T14:06:08.353 INFO:tasks.rados.rados.0.smithi017.stdout:882: done (2 left) 2023-03-29T14:06:08.353 INFO:tasks.rados.rados.0.smithi017.stdout:884: done (1 left) 2023-03-29T14:06:08.353 INFO:tasks.rados.rados.0.smithi017.stdout:885: done (0 left) 2023-03-29T14:06:08.353 INFO:tasks.rados.rados.0.smithi017.stdout:886: write (excl) oid 2 current snap is 99 2023-03-29T14:06:08.353 INFO:tasks.rados.rados.0.smithi017.stdout:886: seq_num 436 ranges {527852=640685,1680502=589135,2757013=570839} 2023-03-29T14:06:08.358 INFO:tasks.rados.rados.0.smithi017.stdout:886: writing smithi017107884-2 from 527852 to 1168537 tid 1 2023-03-29T14:06:08.364 INFO:tasks.rados.rados.0.smithi017.stdout:886: writing smithi017107884-2 from 1680502 to 2269637 tid 2 2023-03-29T14:06:08.371 INFO:tasks.rados.rados.0.smithi017.stdout:886: writing smithi017107884-2 from 2757013 to 3327852 tid 3 2023-03-29T14:06:08.371 INFO:tasks.rados.rados.0.smithi017.stdout:887: write oid 24 current snap is 99 2023-03-29T14:06:08.371 INFO:tasks.rados.rados.0.smithi017.stdout:887: seq_num 437 ranges {544659=715934,1738984=449146,2784436=560223} 2023-03-29T14:06:08.378 INFO:tasks.rados.rados.0.smithi017.stdout:887: writing smithi017107884-24 from 544659 to 1260593 tid 1 2023-03-29T14:06:08.383 INFO:tasks.rados.rados.0.smithi017.stdout:887: writing smithi017107884-24 from 1738984 to 2188130 tid 2 2023-03-29T14:06:08.390 INFO:tasks.rados.rados.0.smithi017.stdout:887: writing smithi017107884-24 from 2784436 to 3344659 tid 3 2023-03-29T14:06:08.390 INFO:tasks.rados.rados.0.smithi017.stdout:888: delete oid 1 current snap is 99 2023-03-29T14:06:08.449 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:08.446+0000 7f5ed7c2e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:08.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:08.453+0000 7f5ed7c2e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:08.477 INFO:tasks.rados.rados.0.smithi017.stdout:886: finishing write tid 1 to smithi017107884-2 2023-03-29T14:06:08.517 INFO:tasks.rados.rados.0.smithi017.stdout:886: finishing write tid 2 to smithi017107884-2 2023-03-29T14:06:08.528 INFO:tasks.rados.rados.0.smithi017.stdout:886: finishing write tid 3 to smithi017107884-2 2023-03-29T14:06:08.532 INFO:tasks.rados.rados.0.smithi017.stdout:887: finishing write tid 1 to smithi017107884-24 2023-03-29T14:06:08.543 INFO:tasks.rados.rados.0.smithi017.stdout:886: finishing write tid 4 to smithi017107884-2 2023-03-29T14:06:08.547 INFO:tasks.rados.rados.0.smithi017.stdout:887: finishing write tid 2 to smithi017107884-24 2023-03-29T14:06:08.555 INFO:tasks.rados.rados.0.smithi017.stdout:886: finishing write tid 5 to smithi017107884-2 2023-03-29T14:06:08.555 INFO:tasks.rados.rados.0.smithi017.stdout:886: oid 2 updating version 0 to 190 2023-03-29T14:06:08.555 INFO:tasks.rados.rados.0.smithi017.stdout:886: oid 2 version 190 is already newer than 190 2023-03-29T14:06:08.555 INFO:tasks.rados.rados.0.smithi017.stdout:886: oid 2 version 190 is already newer than 187 2023-03-29T14:06:08.555 INFO:tasks.rados.rados.0.smithi017.stdout:886: oid 2 version 190 is already newer than 190 2023-03-29T14:06:08.555 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 190 (ObjNum 436 snap 99 seq_num 436) dirty exists 2023-03-29T14:06:08.555 INFO:tasks.rados.rados.0.smithi017.stdout:886: left oid 2 (ObjNum 436 snap 99 seq_num 436) 2023-03-29T14:06:08.566 INFO:tasks.rados.rados.0.smithi017.stdout:887: finishing write tid 3 to smithi017107884-24 2023-03-29T14:06:08.575 INFO:tasks.rados.rados.0.smithi017.stdout:887: finishing write tid 4 to smithi017107884-24 2023-03-29T14:06:08.578 INFO:tasks.rados.rados.0.smithi017.stdout:887: finishing write tid 5 to smithi017107884-24 2023-03-29T14:06:08.579 INFO:tasks.rados.rados.0.smithi017.stdout:887: oid 24 updating version 0 to 77 2023-03-29T14:06:08.579 INFO:tasks.rados.rados.0.smithi017.stdout:887: oid 24 updating version 77 to 78 2023-03-29T14:06:08.579 INFO:tasks.rados.rados.0.smithi017.stdout:887: oid 24 version 78 is already newer than 76 2023-03-29T14:06:08.579 INFO:tasks.rados.rados.0.smithi017.stdout:887: oid 24 version 78 is already newer than 78 2023-03-29T14:06:08.579 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 78 (ObjNum 437 snap 99 seq_num 437) dirty exists 2023-03-29T14:06:08.579 INFO:tasks.rados.rados.0.smithi017.stdout:887: left oid 24 (ObjNum 437 snap 99 seq_num 437) 2023-03-29T14:06:08.617 INFO:tasks.rados.rados.0.smithi017.stdout:886: done (2 left) 2023-03-29T14:06:08.618 INFO:tasks.rados.rados.0.smithi017.stdout:887: done (1 left) 2023-03-29T14:06:08.618 INFO:tasks.rados.rados.0.smithi017.stdout:888: done (0 left) 2023-03-29T14:06:08.618 INFO:tasks.rados.rados.0.smithi017.stdout:889: write oid 16 current snap is 99 2023-03-29T14:06:08.618 INFO:tasks.rados.rados.0.smithi017.stdout:889: seq_num 439 ranges {578273=466432,1739595=452815,2722929=655344} 2023-03-29T14:06:08.623 INFO:tasks.rados.rados.0.smithi017.stdout:889: writing smithi017107884-16 from 578273 to 1044705 tid 1 2023-03-29T14:06:08.629 INFO:tasks.rados.rados.0.smithi017.stdout:889: writing smithi017107884-16 from 1739595 to 2192410 tid 2 2023-03-29T14:06:08.636 INFO:tasks.rados.rados.0.smithi017.stdout:889: writing smithi017107884-16 from 2722929 to 3378273 tid 3 2023-03-29T14:06:08.636 INFO:tasks.rados.rados.0.smithi017.stdout:890: write (excl) oid 23 current snap is 99 2023-03-29T14:06:08.636 INFO:tasks.rados.rados.0.smithi017.stdout:890: seq_num 440 ranges {595080=541681,1798077=596473,3150352=244728} 2023-03-29T14:06:08.643 INFO:tasks.rados.rados.0.smithi017.stdout:890: writing smithi017107884-23 from 595080 to 1136761 tid 1 2023-03-29T14:06:08.650 INFO:tasks.rados.rados.0.smithi017.stdout:890: writing smithi017107884-23 from 1798077 to 2394550 tid 2 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:890: writing smithi017107884-23 from 3150352 to 3395080 tid 3 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:891: read oid 26 snap -1 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:891: expect (ObjNum 365 snap 90 seq_num 365) 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:892: read oid 15 snap -1 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:892: expect (ObjNum 373 snap 90 seq_num 373) 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:893: write oid 6 current snap is 99 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:889: finishing write tid 1 to smithi017107884-16 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:889: finishing write tid 2 to smithi017107884-16 2023-03-29T14:06:08.656 INFO:tasks.rados.rados.0.smithi017.stdout:893: seq_num 441 ranges {611887=733283,1972912=456484,2894128=517759} 2023-03-29T14:06:08.664 INFO:tasks.rados.rados.0.smithi017.stdout:893: writing smithi017107884-6 from 611887 to 1345170 tid 1 2023-03-29T14:06:08.670 INFO:tasks.rados.rados.0.smithi017.stdout:893: writing smithi017107884-6 from 1972912 to 2429396 tid 2 2023-03-29T14:06:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:08.676 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:08.676 INFO:tasks.rados.rados.0.smithi017.stdout:893: writing smithi017107884-6 from 2894128 to 3411887 tid 3 2023-03-29T14:06:08.676 INFO:tasks.rados.rados.0.smithi017.stdout:894: read oid 34 snap -1 2023-03-29T14:06:08.676 INFO:tasks.rados.rados.0.smithi017.stdout:894: expect (ObjNum 424 snap 98 seq_num 424) 2023-03-29T14:06:08.676 INFO:tasks.rados.rados.0.smithi017.stdout:895: snap_create 2023-03-29T14:06:08.676 INFO:tasks.rados.rados.0.smithi017.stdout:891: expect (ObjNum 365 snap 90 seq_num 365) 2023-03-29T14:06:08.689 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2023-03-29T14:06:08.214901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":177657998,"num_objects":504,"num_object_clones":408,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":2119,"num_read_kb":1167596,"num_write":2824,"num_write_kb":742370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1482,"ondisk_log_size":1482,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1008068,"kb_used_data":952740,"kb_used_omap":0,"kb_used_meta":55296,"kb_avail":376479292,"statfs":{"total":386547056640,"available":385514795008,"internally_reserved":0,"allocated":975605760,"data_stored":1176957398,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":56623104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4941720,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":77795,"num_write":128,"num_write_kb":36776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.930629"},"pg_stats":[{"pgid":"3.f","version":"218'57","reported_seq":81,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:05.310779+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:05.310779+0000","last_peered":"2023-03-29T14:06:05.310779+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:05.310779+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5411986,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":25,"num_read_kb":7957,"num_write":32,"num_write_kb":8458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"209'21","reported_seq":221,"reported_epoch":219,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:04.535483+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:04.535483+0000","last_peered":"2023-03-29T14:06:04.535483+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:04.535483+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5075846,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":17991,"num_write":44,"num_write_kb":12992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"220'123","reported_seq":450,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-03-29T14:06:06.392546+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:06.392546+0000","last_peered":"2023-03-29T14:06:06.392546+0000","last_clean":"2023-03-29T14:06:06.392546+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:06.392546+0000","last_undegraded":"2023-03-29T14:06:06.392546+0000","last_fullsized":"2023-03-29T14:06:06.392546+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17076308,"num_objects":40,"num_object_clones":30,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":167,"num_read_kb":113922,"num_write":236,"num_write_kb":69020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"219'140","reported_seq":489,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-03-29T14:06:05.801371+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:05.801371+0000","last_peered":"2023-03-29T14:06:05.801371+0000","last_clean":"2023-03-29T14:06:05.801371+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:05.801371+0000","last_undegraded":"2023-03-29T14:06:05.801371+0000","last_fullsized":"2023-03-29T14:06:05.801371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18353640,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":205,"num_read_kb":118718,"num_write":274,"num_write_kb":76938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"219'119","reported_seq":437,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-03-29T14:06:05.782491+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:05.782491+0000","last_peered":"2023-03-29T14:06:05.782491+0000","last_clean":"2023-03-29T14:06:05.782491+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:05.782491+0000","last_undegraded":"2023-03-29T14:06:05.782491+0000","last_fullsized":"2023-03-29T14:06:05.782491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24202608,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":165,"num_read_kb":94308,"num_write":242,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":180,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-03-29T14:06:05.315008+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:05.315008+0000","last_peered":"2023-03-29T14:06:05.315008+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:05.315008+0000","last_undegraded":"2023-03-29T14:06:05.315008+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"213'86","reported_seq":363,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:05.301628+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:05.301628+0000","last_peered":"2023-03-29T14:06:05.301628+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:05.301628+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7596830,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":108,"num_read_kb":51657,"num_write":154,"num_write_kb":45076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"216'59","reported_seq":309,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-03-29T14:06:05.388363+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:05.388363+0000","last_peered":"2023-03-29T14:06:05.388363+0000","last_clean":"2023-03-29T14:06:05.388363+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:05.388363+0000","last_undegraded":"2023-03-29T14:06:05.388363+0000","last_fullsized":"2023-03-29T14:06:05.388363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":86,"num_read_kb":49141,"num_write":116,"num_write_kb":32686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":219,"state":"active+undersized","last_fresh":"2023-03-29T14:06:04.541304+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:04.541304+0000","last_peered":"2023-03-29T14:06:04.541304+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:04.541304+0000","last_undegraded":"2023-03-29T14:06:04.541304+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":48,"reported_epoch":219,"state":"active+undersized","last_fresh":"2023-03-29T14:06:04.699630+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:04.699630+0000","last_peered":"2023-03-29T14:06:04.699630+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:04.699630+0000","last_undegraded":"2023-03-29T14:06:04.699630+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7262,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":183,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-03-29T14:06:05.314228+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:05.314228+0000","last_peered":"2023-03-29T14:06:05.314228+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:05.314228+0000","last_undegraded":"2023-03-29T14:06:05.314228+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"219'72","reported_seq":339,"reported_epoch":219,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:05.686686+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:05.686686+0000","last_peered":"2023-03-29T14:06:05.686686+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:05.686686+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12370150,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":70993,"num_write":146,"num_write_kb":41570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":219,"state":"active+undersized","last_fresh":"2023-03-29T14:06:04.549685+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:04.549685+0000","last_peered":"2023-03-29T14:06:04.549685+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:04.549685+0000","last_undegraded":"2023-03-29T14:06:04.549685+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"219'87","reported_seq":366,"reported_epoch":219,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:05.632058+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:05.632058+0000","last_peered":"2023-03-29T14:06:05.632058+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:05.632058+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13950008,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":106,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":183,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-03-29T14:06:05.299757+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:05.299757+0000","last_peered":"2023-03-29T14:06:05.299757+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:05.299757+0000","last_undegraded":"2023-03-29T14:06:05.299757+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":401,"reported_epoch":219,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:04.542585+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:04.542585+0000","last_peered":"2023-03-29T14:06:04.542585+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:04.542585+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"213'80","reported_seq":397,"reported_epoch":220,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:05.317857+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:05.317857+0000","last_peered":"2023-03-29T14:06:05.317857+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:05.317857+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15597226,"num_objects":32,"num_object_clones":24,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":195,"num_read_kb":126317,"num_write":158,"num_write_kb":44220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":32}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":183,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-03-29T14:06:05.304054+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:05.304054+0000","last_peered":"2023-03-29T14:06:05.304054+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:05.304054+0000","last_undegraded":"2023-03-29T14:06:05.304054+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"219'186","reported_seq":603,"reported_epoch":219,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:05.830613+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:05.830613+0000","last_peered":"2023-03-29T14:06:05.830613+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:05.830613+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16269572,"num_objects":68,"num_object_clones":54,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":309,"num_read_kb":190744,"num_write":344,"num_write_kb":99234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":176,"reported_epoch":219,"state":"active+undersized","last_fresh":"2023-03-29T14:06:04.538463+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:04.538463+0000","last_peered":"2023-03-29T14:06:04.538463+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:04.538463+0000","last_undegraded":"2023-03-29T14:06:04.538463+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"202'63","reported_seq":321,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-03-29T14:06:04.582692+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:04.582692+0000","last_peered":"2023-03-29T14:06:04.582692+0000","last_clean":"2023-03-29T14:06:04.582692+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:04.582692+0000","last_undegraded":"2023-03-29T14:06:04.582692+0000","last_fullsized":"2023-03-29T14:06:04.582692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8639062,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":59594,"num_write":134,"num_write_kb":37726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"214'136","reported_seq":466,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-03-29T14:06:04.527500+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:04.527500+0000","last_peered":"2023-03-29T14:06:04.527500+0000","last_clean":"2023-03-29T14:06:04.527500+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:04.527500+0000","last_undegraded":"2023-03-29T14:06:04.527500+0000","last_fullsized":"2023-03-29T14:06:04.527500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723130,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":173,"num_read_kb":98003,"num_write":260,"num_write_kb":76742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":176,"reported_epoch":219,"state":"active+undersized","last_fresh":"2023-03-29T14:06:04.512241+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:04.512241+0000","last_peered":"2023-03-29T14:06:04.512241+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:04.512241+0000","last_undegraded":"2023-03-29T14:06:04.512241+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":183,"reported_epoch":220,"state":"active+undersized","last_fresh":"2023-03-29T14:06:05.311582+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:05.311582+0000","last_peered":"2023-03-29T14:06:05.311582+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:05.311582+0000","last_undegraded":"2023-03-29T14:06:05.311582+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"217'114","reported_seq":437,"reported_epoch":219,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:05.747750+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:05.747750+0000","last_peered":"2023-03-29T14:06:05.747750+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:05.747750+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773716,"num_objects":42,"num_object_clones":34,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":180,"num_read_kb":109996,"num_write":224,"num_write_kb":65854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":176477262,"num_objects":500,"num_object_clones":408,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":1977,"num_read_kb":1167476,"num_write":2550,"num_write_kb":738582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1385,"ondisk_log_size":1385,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":272136,"kb_used_data":258240,"kb_used_omap":0,"kb_used_meta":13888,"kb_avail":94099704,"statfs":{"total":96636764160,"available":96358096896,"internally_reserved":0,"allocated":264437760,"data_stored":316443606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14221312},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":300432,"kb_used_data":285576,"kb_used_omap":0,"kb_used_meta":14848,"kb_avail":94071408,"statfs":{"total":96636764160,"available":96329121792,"internally_reserved":0,"allocated":292429824,"data_stored":356181631,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15204352},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":279556,"kb_used_data":265084,"kb_used_omap":0,"kb_used_meta":14464,"kb_avail":94092284,"statfs":{"total":96636764160,"available":96350498816,"internally_reserved":0,"allocated":271446016,"data_stored":331099936,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":14811136},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:08.761 INFO:tasks.rados.rados.0.smithi017.stdout:889: finishing write tid 3 to smithi017107884-16 2023-03-29T14:06:08.761 INFO:tasks.rados.rados.0.smithi017.stdout:890: finishing write tid 1 to smithi017107884-23 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:889: finishing write tid 4 to smithi017107884-16 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:889: finishing write tid 5 to smithi017107884-16 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:889: oid 16 updating version 0 to 66 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:889: oid 16 version 66 is already newer than 64 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:889: oid 16 version 66 is already newer than 66 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:889: oid 16 version 66 is already newer than 66 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 66 (ObjNum 439 snap 99 seq_num 439) dirty exists 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:889: left oid 16 (ObjNum 439 snap 99 seq_num 439) 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:893: finishing write tid 1 to smithi017107884-6 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:893: finishing write tid 2 to smithi017107884-6 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:890: finishing write tid 2 to smithi017107884-23 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:890: finishing write tid 3 to smithi017107884-23 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:890: finishing write tid 4 to smithi017107884-23 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:890: finishing write tid 5 to smithi017107884-23 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:890: oid 23 updating version 0 to 191 2023-03-29T14:06:08.762 INFO:tasks.rados.rados.0.smithi017.stdout:890: oid 23 updating version 191 to 194 2023-03-29T14:06:08.763 INFO:tasks.rados.rados.0.smithi017.stdout:890: oid 23 version 194 is already newer than 194 2023-03-29T14:06:08.763 INFO:tasks.rados.rados.0.smithi017.stdout:890: oid 23 version 194 is already newer than 194 2023-03-29T14:06:08.763 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 194 (ObjNum 440 snap 99 seq_num 440) dirty exists 2023-03-29T14:06:08.763 INFO:tasks.rados.rados.0.smithi017.stdout:890: left oid 23 (ObjNum 440 snap 99 seq_num 440) 2023-03-29T14:06:08.763 INFO:tasks.rados.rados.0.smithi017.stdout:894: expect (ObjNum 424 snap 98 seq_num 424) 2023-03-29T14:06:08.978 INFO:tasks.rados.rados.0.smithi017.stdout:892: expect (ObjNum 373 snap 90 seq_num 373) 2023-03-29T14:06:09.094 INFO:tasks.rados.rados.0.smithi017.stdout:893: finishing write tid 3 to smithi017107884-6 2023-03-29T14:06:09.094 INFO:tasks.rados.rados.0.smithi017.stdout:893: finishing write tid 4 to smithi017107884-6 2023-03-29T14:06:09.094 INFO:tasks.rados.rados.0.smithi017.stdout:893: finishing write tid 5 to smithi017107884-6 2023-03-29T14:06:09.095 INFO:tasks.rados.rados.0.smithi017.stdout:893: oid 6 updating version 0 to 84 2023-03-29T14:06:09.095 INFO:tasks.rados.rados.0.smithi017.stdout:893: oid 6 version 84 is already newer than 84 2023-03-29T14:06:09.095 INFO:tasks.rados.rados.0.smithi017.stdout:893: oid 6 version 84 is already newer than 84 2023-03-29T14:06:09.095 INFO:tasks.rados.rados.0.smithi017.stdout:893: oid 6 version 84 is already newer than 84 2023-03-29T14:06:09.095 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 84 (ObjNum 441 snap 99 seq_num 441) dirty exists 2023-03-29T14:06:09.095 INFO:tasks.rados.rados.0.smithi017.stdout:893: left oid 6 (ObjNum 441 snap 99 seq_num 441) 2023-03-29T14:06:09.271 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:09.304 INFO:tasks.rados.rados.0.smithi017.stdout:889: done (6 left) 2023-03-29T14:06:09.304 INFO:tasks.rados.rados.0.smithi017.stdout:890: done (5 left) 2023-03-29T14:06:09.304 INFO:tasks.rados.rados.0.smithi017.stdout:891: done (4 left) 2023-03-29T14:06:09.304 INFO:tasks.rados.rados.0.smithi017.stdout:892: done (3 left) 2023-03-29T14:06:09.305 INFO:tasks.rados.rados.0.smithi017.stdout:893: done (2 left) 2023-03-29T14:06:09.305 INFO:tasks.rados.rados.0.smithi017.stdout:894: done (1 left) 2023-03-29T14:06:09.305 INFO:tasks.rados.rados.0.smithi017.stdout:895: done (0 left) 2023-03-29T14:06:09.305 INFO:tasks.rados.rados.0.smithi017.stdout:896: snap_create 2023-03-29T14:06:09.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:09.402+0000 7f9cd8687700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:09.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:09.407+0000 7f9cd8687700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:09.469 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:09.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:09.616 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:10.307 INFO:tasks.rados.rados.0.smithi017.stdout:896: done (0 left) 2023-03-29T14:06:10.307 INFO:tasks.rados.rados.0.smithi017.stdout:897: delete oid 13 current snap is 101 2023-03-29T14:06:10.383 INFO:tasks.rados.rados.0.smithi017.stdout:897: done (0 left) 2023-03-29T14:06:10.383 INFO:tasks.rados.rados.0.smithi017.stdout:898: delete oid 45 current snap is 101 2023-03-29T14:06:10.437 INFO:tasks.rados.rados.0.smithi017.stdout:898: done (0 left) 2023-03-29T14:06:10.438 INFO:tasks.rados.rados.0.smithi017.stdout:899: delete oid 48 current snap is 101 2023-03-29T14:06:10.494 INFO:tasks.rados.rados.0.smithi017.stdout:899: done (0 left) 2023-03-29T14:06:10.495 INFO:tasks.rados.rados.0.smithi017.stdout:900: read oid 11 snap -1 2023-03-29T14:06:10.495 INFO:tasks.rados.rados.0.smithi017.stdout:900: expect (ObjNum 395 snap 94 seq_num 395) 2023-03-29T14:06:10.495 INFO:tasks.rados.rados.0.smithi017.stdout:901: read oid 46 snap -1 2023-03-29T14:06:10.495 INFO:tasks.rados.rados.0.smithi017.stdout:901: expect (ObjNum 411 snap 97 seq_num 411) 2023-03-29T14:06:10.495 INFO:tasks.rados.rados.0.smithi017.stdout:902: read oid 16 snap -1 2023-03-29T14:06:10.495 INFO:tasks.rados.rados.0.smithi017.stdout:902: expect (ObjNum 439 snap 99 seq_num 439) 2023-03-29T14:06:10.495 INFO:tasks.rados.rados.0.smithi017.stdout:903: snap_create 2023-03-29T14:06:10.512 INFO:tasks.rados.rados.0.smithi017.stdout:902: expect (ObjNum 439 snap 99 seq_num 439) 2023-03-29T14:06:10.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~61s 2023-03-29T14:06:10.644 INFO:tasks.rados.rados.0.smithi017.stdout:900: expect (ObjNum 395 snap 94 seq_num 395) 2023-03-29T14:06:10.747 INFO:tasks.rados.rados.0.smithi017.stdout:901: expect (ObjNum 411 snap 97 seq_num 411) 2023-03-29T14:06:11.311 INFO:tasks.rados.rados.0.smithi017.stdout:900: done (3 left) 2023-03-29T14:06:11.311 INFO:tasks.rados.rados.0.smithi017.stdout:901: done (2 left) 2023-03-29T14:06:11.311 INFO:tasks.rados.rados.0.smithi017.stdout:902: done (1 left) 2023-03-29T14:06:11.311 INFO:tasks.rados.rados.0.smithi017.stdout:903: done (0 left) 2023-03-29T14:06:11.311 INFO:tasks.rados.rados.0.smithi017.stdout:904: delete oid 49 current snap is 102 2023-03-29T14:06:11.423 INFO:tasks.rados.rados.0.smithi017.stdout:904: done (0 left) 2023-03-29T14:06:11.423 INFO:tasks.rados.rados.0.smithi017.stdout:905: write oid 8 current snap is 102 2023-03-29T14:06:11.423 INFO:tasks.rados.rados.0.smithi017.stdout:905: seq_num 446 ranges {695922=709528,2032616=607480,3198537=297385} 2023-03-29T14:06:11.430 INFO:tasks.rados.rados.0.smithi017.stdout:905: writing smithi017107884-8 from 695922 to 1405450 tid 1 2023-03-29T14:06:11.437 INFO:tasks.rados.rados.0.smithi017.stdout:905: writing smithi017107884-8 from 2032616 to 2640096 tid 2 2023-03-29T14:06:11.442 INFO:tasks.rados.rados.0.smithi017.stdout:905: writing smithi017107884-8 from 3198537 to 3495922 tid 3 2023-03-29T14:06:11.442 INFO:tasks.rados.rados.0.smithi017.stdout:906: write (excl) oid 46 current snap is 102 2023-03-29T14:06:11.442 INFO:tasks.rados.rados.0.smithi017.stdout:906: seq_num 447 ranges {712729=784777,2091098=467491,3342313=170416} 2023-03-29T14:06:11.450 INFO:tasks.rados.rados.0.smithi017.stdout:906: writing smithi017107884-46 from 712729 to 1497506 tid 1 2023-03-29T14:06:11.456 INFO:tasks.rados.rados.0.smithi017.stdout:906: writing smithi017107884-46 from 2091098 to 2558589 tid 2 2023-03-29T14:06:11.461 INFO:tasks.rados.rados.0.smithi017.stdout:906: writing smithi017107884-46 from 3342313 to 3512729 tid 3 2023-03-29T14:06:11.461 INFO:tasks.rados.rados.0.smithi017.stdout:907: snap_remove snap 101 2023-03-29T14:06:11.617 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:11.749+0000 7f0b23906700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:11.754+0000 7f0b23906700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:11.817 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:11.820 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:11.820 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:12.315 INFO:tasks.rados.rados.0.smithi017.stdout:907: done (2 left) 2023-03-29T14:06:12.315 INFO:tasks.rados.rados.0.smithi017.stdout:908: snap_remove snap 100 2023-03-29T14:06:13.316 INFO:tasks.rados.rados.0.smithi017.stdout:908: done (2 left) 2023-03-29T14:06:13.316 INFO:tasks.rados.rados.0.smithi017.stdout:909: read oid 7 snap -1 2023-03-29T14:06:13.316 INFO:tasks.rados.rados.0.smithi017.stdout:909: expect deleted 2023-03-29T14:06:13.316 INFO:tasks.rados.rados.0.smithi017.stdout:910: snap_remove snap 99 2023-03-29T14:06:13.693 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:13.693 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:13.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:13.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:13.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:13.823+0000 7fa7d62e9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:13.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:13.830+0000 7fa7d62e9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:13.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:13.951+0000 7f51911e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:13.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:13.956+0000 7f51911e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:14.021 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:14.022 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:14.041 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:14.055 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2023-03-29T14:06:13.314851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":182061564,"num_objects":510,"num_object_clones":414,"num_object_copies":1530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":2181,"num_read_kb":1202634,"num_write":2894,"num_write_kb":764268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1516,"ondisk_log_size":1516,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1046792,"kb_used_data":990496,"kb_used_omap":0,"kb_used_meta":55872,"kb_avail":376440568,"statfs":{"total":386547056640,"available":385475141632,"internally_reserved":0,"allocated":1014267904,"data_stored":1218852509,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":57212928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4403566,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35038,"num_write":70,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012096"},"pg_stats":[{"pgid":"3.f","version":"222'66","reported_seq":107,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:10.505038+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:10.505038+0000","last_peered":"2023-03-29T14:06:10.505038+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:10.505038+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345022,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":14335,"num_write":52,"num_write_kb":14796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"209'21","reported_seq":225,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:09.342233+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:09.342233+0000","last_peered":"2023-03-29T14:06:09.342233+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:09.342233+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5075846,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":17991,"num_write":44,"num_write_kb":12992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"220'123","reported_seq":456,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-03-29T14:06:10.473441+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:10.473441+0000","last_peered":"2023-03-29T14:06:10.473441+0000","last_clean":"2023-03-29T14:06:10.473441+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:10.473441+0000","last_undegraded":"2023-03-29T14:06:10.473441+0000","last_fullsized":"2023-03-29T14:06:10.473441+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10588674,"num_objects":42,"num_object_clones":32,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":169,"num_read_kb":113922,"num_write":236,"num_write_kb":69020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"221'144","reported_seq":502,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-03-29T14:06:09.518201+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:09.518201+0000","last_peered":"2023-03-29T14:06:09.518201+0000","last_clean":"2023-03-29T14:06:09.518201+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:09.518201+0000","last_undegraded":"2023-03-29T14:06:09.518201+0000","last_fullsized":"2023-03-29T14:06:09.518201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521710,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":208,"num_read_kb":118719,"num_write":284,"num_write_kb":79564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"224'121","reported_seq":449,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-03-29T14:06:10.518567+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:10.518567+0000","last_peered":"2023-03-29T14:06:10.518567+0000","last_clean":"2023-03-29T14:06:10.518567+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:10.518567+0000","last_undegraded":"2023-03-29T14:06:10.518567+0000","last_fullsized":"2023-03-29T14:06:10.518567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18420868,"num_objects":48,"num_object_clones":40,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":179,"num_read_kb":110004,"num_write":242,"num_write_kb":72320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":184,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.526886+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:09.526886+0000","last_peered":"2023-03-29T14:06:09.526886+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:09.526886+0000","last_undegraded":"2023-03-29T14:06:09.526886+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"213'86","reported_seq":367,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:09.520989+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:09.520989+0000","last_peered":"2023-03-29T14:06:09.520989+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:09.520989+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7596830,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":108,"num_read_kb":51657,"num_write":154,"num_write_kb":45076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"216'59","reported_seq":313,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-03-29T14:06:09.516622+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:09.516622+0000","last_peered":"2023-03-29T14:06:09.516622+0000","last_clean":"2023-03-29T14:06:09.516622+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:09.516622+0000","last_undegraded":"2023-03-29T14:06:09.516622+0000","last_fullsized":"2023-03-29T14:06:09.516622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":86,"num_read_kb":49141,"num_write":116,"num_write_kb":32686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.351359+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:09.351359+0000","last_peered":"2023-03-29T14:06:09.351359+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:09.351359+0000","last_undegraded":"2023-03-29T14:06:09.351359+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":52,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.605530+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:09.605530+0000","last_peered":"2023-03-29T14:06:09.605530+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:09.605530+0000","last_undegraded":"2023-03-29T14:06:09.605530+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7262,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":187,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.515371+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:09.515371+0000","last_peered":"2023-03-29T14:06:09.515371+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:09.515371+0000","last_undegraded":"2023-03-29T14:06:09.515371+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"222'78","reported_seq":354,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:09.590667+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:09.590667+0000","last_peered":"2023-03-29T14:06:09.590667+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:09.590667+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13109724,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":70994,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.354486+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:09.354486+0000","last_peered":"2023-03-29T14:06:09.354486+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:09.354486+0000","last_undegraded":"2023-03-29T14:06:09.354486+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"219'87","reported_seq":370,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:09.378375+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:09.378375+0000","last_peered":"2023-03-29T14:06:09.378375+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:09.378375+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13950008,"num_objects":32,"num_object_clones":26,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":106,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":187,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.521736+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:09.521736+0000","last_peered":"2023-03-29T14:06:09.521736+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:09.521736+0000","last_undegraded":"2023-03-29T14:06:09.521736+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":405,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:09.340352+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:09.340352+0000","last_peered":"2023-03-29T14:06:09.340352+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:09.340352+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"222'84","reported_seq":413,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:09.524556+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:09.524556+0000","last_peered":"2023-03-29T14:06:09.524556+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:09.524556+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15832590,"num_objects":32,"num_object_clones":24,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":203,"num_read_kb":131588,"num_write":168,"num_write_kb":47558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":32}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.511966+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:09.511966+0000","last_peered":"2023-03-29T14:06:09.511966+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:09.511966+0000","last_undegraded":"2023-03-29T14:06:09.511966+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"222'194","reported_seq":631,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:10.511081+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:10.511081+0000","last_peered":"2023-03-29T14:06:10.511081+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:10.511081+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29715436,"num_objects":68,"num_object_clones":54,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":329,"num_read_kb":198435,"num_write":364,"num_write_kb":105456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":180,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.366691+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:09.366691+0000","last_peered":"2023-03-29T14:06:09.366691+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:09.366691+0000","last_undegraded":"2023-03-29T14:06:09.366691+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"202'63","reported_seq":325,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-03-29T14:06:09.352677+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:09.352677+0000","last_peered":"2023-03-29T14:06:09.352677+0000","last_clean":"2023-03-29T14:06:09.352677+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:09.352677+0000","last_undegraded":"2023-03-29T14:06:09.352677+0000","last_fullsized":"2023-03-29T14:06:09.352677+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8639062,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":59594,"num_write":134,"num_write_kb":37726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"214'136","reported_seq":471,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-03-29T14:06:10.338566+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:10.338566+0000","last_peered":"2023-03-29T14:06:10.338566+0000","last_clean":"2023-03-29T14:06:10.338566+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:10.338566+0000","last_undegraded":"2023-03-29T14:06:10.338566+0000","last_fullsized":"2023-03-29T14:06:10.338566+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723130,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":173,"num_read_kb":98003,"num_write":260,"num_write_kb":76742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":180,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.360297+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:09.360297+0000","last_peered":"2023-03-29T14:06:09.360297+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:09.360297+0000","last_undegraded":"2023-03-29T14:06:09.360297+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":187,"reported_epoch":224,"state":"active+undersized","last_fresh":"2023-03-29T14:06:09.522513+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:09.522513+0000","last_peered":"2023-03-29T14:06:09.522513+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:09.522513+0000","last_undegraded":"2023-03-29T14:06:09.522513+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"222'115","reported_seq":443,"reported_epoch":224,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:09.610455+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:09.610455+0000","last_peered":"2023-03-29T14:06:09.610455+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:09.610455+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10924748,"num_objects":42,"num_object_clones":34,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":180,"num_read_kb":109996,"num_write":224,"num_write_kb":65854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":180880828,"num_objects":506,"num_object_clones":414,"num_object_copies":1518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":2039,"num_read_kb":1202514,"num_write":2620,"num_write_kb":760480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1419,"ondisk_log_size":1419,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":280932,"kb_used_data":266908,"kb_used_omap":0,"kb_used_meta":14016,"kb_avail":94090908,"statfs":{"total":96636764160,"available":96349089792,"internally_reserved":0,"allocated":273313792,"data_stored":324777014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14352384},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":317764,"kb_used_data":302652,"kb_used_omap":0,"kb_used_meta":15104,"kb_avail":94054076,"statfs":{"total":96636764160,"available":96311373824,"internally_reserved":0,"allocated":309915648,"data_stored":375467369,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15466496},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":292152,"kb_used_data":277096,"kb_used_omap":0,"kb_used_meta":14656,"kb_avail":94079688,"statfs":{"total":96636764160,"available":96337600512,"internally_reserved":0,"allocated":283746304,"data_stored":345375901,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15007744},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:14.322 INFO:tasks.rados.rados.0.smithi017.stdout:910: done (3 left) 2023-03-29T14:06:14.322 INFO:tasks.rados.rados.0.smithi017.stdout:911: write oid 28 current snap is 102 2023-03-29T14:06:14.322 INFO:tasks.rados.rados.0.smithi017.stdout:911: seq_num 448 ranges {729536=460026,1749580=611149,2853383=452450} 2023-03-29T14:06:14.328 INFO:tasks.rados.rados.0.smithi017.stdout:911: writing smithi017107884-28 from 729536 to 1189562 tid 1 2023-03-29T14:06:14.335 INFO:tasks.rados.rados.0.smithi017.stdout:911: writing smithi017107884-28 from 1749580 to 2360729 tid 2 2023-03-29T14:06:14.340 INFO:tasks.rados.rados.0.smithi017.stdout:911: writing smithi017107884-28 from 2853383 to 3305833 tid 3 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:912: read oid 23 snap -1 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:912: expect (ObjNum 440 snap 99 seq_num 440) 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:913: delete oid 34 current snap is 102 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: finishing write tid 1 to smithi017107884-8 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: finishing write tid 2 to smithi017107884-8 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: finishing write tid 3 to smithi017107884-8 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: finishing write tid 4 to smithi017107884-8 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: finishing write tid 5 to smithi017107884-8 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: oid 8 updating version 0 to 119 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: oid 8 version 119 is already newer than 119 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: oid 8 version 119 is already newer than 119 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: oid 8 version 119 is already newer than 117 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 119 (ObjNum 446 snap 102 seq_num 446) dirty exists 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:905: left oid 8 (ObjNum 446 snap 102 seq_num 446) 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:906: finishing write tid 1 to smithi017107884-46 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:906: finishing write tid 2 to smithi017107884-46 2023-03-29T14:06:14.341 INFO:tasks.rados.rados.0.smithi017.stdout:906: finishing write tid 3 to smithi017107884-46 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:906: finishing write tid 4 to smithi017107884-46 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:906: finishing write tid 5 to smithi017107884-46 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:906: oid 46 updating version 0 to 126 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:906: oid 46 version 126 is already newer than 126 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:906: oid 46 version 126 is already newer than 126 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:906: oid 46 version 126 is already newer than 122 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 126 (ObjNum 447 snap 102 seq_num 447) dirty exists 2023-03-29T14:06:14.342 INFO:tasks.rados.rados.0.smithi017.stdout:906: left oid 46 (ObjNum 447 snap 102 seq_num 447) 2023-03-29T14:06:14.518 INFO:tasks.rados.rados.0.smithi017.stdout:911: finishing write tid 1 to smithi017107884-28 2023-03-29T14:06:14.525 INFO:tasks.rados.rados.0.smithi017.stdout:912: expect (ObjNum 440 snap 99 seq_num 440) 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: finishing write tid 2 to smithi017107884-28 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: finishing write tid 3 to smithi017107884-28 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: finishing write tid 4 to smithi017107884-28 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: finishing write tid 5 to smithi017107884-28 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: oid 28 updating version 0 to 64 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: oid 28 version 64 is already newer than 64 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: oid 28 version 64 is already newer than 64 2023-03-29T14:06:14.661 INFO:tasks.rados.rados.0.smithi017.stdout:911: oid 28 version 64 is already newer than 64 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 64 (ObjNum 448 snap 102 seq_num 448) dirty exists 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:911: left oid 28 (ObjNum 448 snap 102 seq_num 448) 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:905: done (5 left) 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:906: done (4 left) 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:909: done (3 left) 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:911: done (2 left) 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:912: done (1 left) 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:913: done (0 left) 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:914: read oid 22 snap -1 2023-03-29T14:06:14.662 INFO:tasks.rados.rados.0.smithi017.stdout:914: expect (ObjNum 434 snap 99 seq_num 434) 2023-03-29T14:06:14.663 INFO:tasks.rados.rados.0.smithi017.stdout:915: write oid 21 current snap is 102 2023-03-29T14:06:14.663 INFO:tasks.rados.rados.0.smithi017.stdout:915: seq_num 450 ranges {763150=726877,1982897=614818,3024582=474994} 2023-03-29T14:06:14.668 INFO:tasks.rados.rados.0.smithi017.stdout:915: writing smithi017107884-21 from 763150 to 1490027 tid 1 2023-03-29T14:06:14.674 INFO:tasks.rados.rados.0.smithi017.stdout:915: writing smithi017107884-21 from 1982897 to 2597715 tid 2 2023-03-29T14:06:14.680 INFO:tasks.rados.rados.0.smithi017.stdout:915: writing smithi017107884-21 from 3024582 to 3499576 tid 3 2023-03-29T14:06:14.680 INFO:tasks.rados.rados.0.smithi017.stdout:916: read oid 1 snap -1 2023-03-29T14:06:14.680 INFO:tasks.rados.rados.0.smithi017.stdout:916: expect deleted 2023-03-29T14:06:14.680 INFO:tasks.rados.rados.0.smithi017.stdout:917: delete oid 31 current snap is 102 2023-03-29T14:06:14.680 INFO:tasks.rados.rados.0.smithi017.stdout:914: expect (ObjNum 434 snap 99 seq_num 434) 2023-03-29T14:06:14.850 INFO:tasks.rados.rados.0.smithi017.stdout:915: finishing write tid 1 to smithi017107884-21 2023-03-29T14:06:14.850 INFO:tasks.rados.rados.0.smithi017.stdout:915: finishing write tid 2 to smithi017107884-21 2023-03-29T14:06:14.850 INFO:tasks.rados.rados.0.smithi017.stdout:915: finishing write tid 3 to smithi017107884-21 2023-03-29T14:06:14.850 INFO:tasks.rados.rados.0.smithi017.stdout:914: done (3 left) 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:916: done (2 left) 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:917: done (1 left) 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:918: read oid 38 snap -1 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:918: expect (ObjNum 389 snap 93 seq_num 389) 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:919: delete oid 32 current snap is 102 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:915: finishing write tid 4 to smithi017107884-21 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:915: finishing write tid 5 to smithi017107884-21 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:915: oid 21 updating version 0 to 88 2023-03-29T14:06:14.851 INFO:tasks.rados.rados.0.smithi017.stdout:915: oid 21 version 88 is already newer than 88 2023-03-29T14:06:14.852 INFO:tasks.rados.rados.0.smithi017.stdout:915: oid 21 version 88 is already newer than 88 2023-03-29T14:06:14.852 INFO:tasks.rados.rados.0.smithi017.stdout:915: oid 21 version 88 is already newer than 88 2023-03-29T14:06:14.852 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 88 (ObjNum 450 snap 102 seq_num 450) dirty exists 2023-03-29T14:06:14.852 INFO:tasks.rados.rados.0.smithi017.stdout:915: left oid 21 (ObjNum 450 snap 102 seq_num 450) 2023-03-29T14:06:14.854 INFO:tasks.rados.rados.0.smithi017.stdout:915: done (2 left) 2023-03-29T14:06:14.854 INFO:tasks.rados.rados.0.smithi017.stdout:919: done (1 left) 2023-03-29T14:06:14.854 INFO:tasks.rados.rados.0.smithi017.stdout:920: write (excl) oid 20 current snap is 102 2023-03-29T14:06:14.854 INFO:tasks.rados.rados.0.smithi017.stdout:920: seq_num 453 ranges {413571=552624,1641990=762145,2990498=425163} 2023-03-29T14:06:14.859 INFO:tasks.rados.rados.0.smithi017.stdout:920: writing smithi017107884-20 from 413571 to 966195 tid 1 2023-03-29T14:06:14.867 INFO:tasks.rados.rados.0.smithi017.stdout:920: writing smithi017107884-20 from 1641990 to 2404135 tid 2 2023-03-29T14:06:14.873 INFO:tasks.rados.rados.0.smithi017.stdout:920: writing smithi017107884-20 from 2990498 to 3415661 tid 3 2023-03-29T14:06:14.873 INFO:tasks.rados.rados.0.smithi017.stdout:921: write (excl) oid 14 current snap is 102 2023-03-29T14:06:14.873 INFO:tasks.rados.rados.0.smithi017.stdout:921: seq_num 454 ranges {430378=627873,1700472=622156,3017921=612457} 2023-03-29T14:06:14.879 INFO:tasks.rados.rados.0.smithi017.stdout:921: writing smithi017107884-14 from 430378 to 1058251 tid 1 2023-03-29T14:06:14.886 INFO:tasks.rados.rados.0.smithi017.stdout:921: writing smithi017107884-14 from 1700472 to 2322628 tid 2 2023-03-29T14:06:14.894 INFO:tasks.rados.rados.0.smithi017.stdout:921: writing smithi017107884-14 from 3017921 to 3630378 tid 3 2023-03-29T14:06:14.894 INFO:tasks.rados.rados.0.smithi017.stdout:922: snap_create 2023-03-29T14:06:14.903 INFO:tasks.rados.rados.0.smithi017.stdout:918: expect (ObjNum 389 snap 93 seq_num 389) 2023-03-29T14:06:15.112 INFO:tasks.rados.rados.0.smithi017.stdout:920: finishing write tid 1 to smithi017107884-20 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:921: finishing write tid 1 to smithi017107884-14 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:921: finishing write tid 2 to smithi017107884-14 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: finishing write tid 2 to smithi017107884-20 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: finishing write tid 3 to smithi017107884-20 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: finishing write tid 4 to smithi017107884-20 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: finishing write tid 5 to smithi017107884-20 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: oid 20 updating version 0 to 131 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: oid 20 version 131 is already newer than 131 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: oid 20 version 131 is already newer than 128 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: oid 20 version 131 is already newer than 131 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 131 (ObjNum 453 snap 102 seq_num 453) dirty exists 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:920: left oid 20 (ObjNum 453 snap 102 seq_num 453) 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:921: finishing write tid 3 to smithi017107884-14 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:921: finishing write tid 4 to smithi017107884-14 2023-03-29T14:06:15.113 INFO:tasks.rados.rados.0.smithi017.stdout:921: finishing write tid 5 to smithi017107884-14 2023-03-29T14:06:15.114 INFO:tasks.rados.rados.0.smithi017.stdout:921: oid 14 updating version 0 to 88 2023-03-29T14:06:15.114 INFO:tasks.rados.rados.0.smithi017.stdout:921: oid 14 updating version 88 to 91 2023-03-29T14:06:15.114 INFO:tasks.rados.rados.0.smithi017.stdout:921: oid 14 version 91 is already newer than 91 2023-03-29T14:06:15.114 INFO:tasks.rados.rados.0.smithi017.stdout:921: oid 14 version 91 is already newer than 91 2023-03-29T14:06:15.114 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 91 (ObjNum 454 snap 102 seq_num 454) dirty exists 2023-03-29T14:06:15.114 INFO:tasks.rados.rados.0.smithi017.stdout:921: left oid 14 (ObjNum 454 snap 102 seq_num 454) 2023-03-29T14:06:15.325 INFO:tasks.rados.rados.0.smithi017.stdout:918: done (3 left) 2023-03-29T14:06:15.326 INFO:tasks.rados.rados.0.smithi017.stdout:920: done (2 left) 2023-03-29T14:06:15.326 INFO:tasks.rados.rados.0.smithi017.stdout:921: done (1 left) 2023-03-29T14:06:15.326 INFO:tasks.rados.rados.0.smithi017.stdout:922: done (0 left) 2023-03-29T14:06:15.326 INFO:tasks.rados.rados.0.smithi017.stdout:923: read oid 48 snap -1 2023-03-29T14:06:15.326 INFO:tasks.rados.rados.0.smithi017.stdout:923: expect deleted 2023-03-29T14:06:15.326 INFO:tasks.rados.rados.0.smithi017.stdout:924: delete oid 38 current snap is 103 2023-03-29T14:06:15.448 INFO:tasks.rados.rados.0.smithi017.stdout:923: done (1 left) 2023-03-29T14:06:15.448 INFO:tasks.rados.rados.0.smithi017.stdout:924: done (0 left) 2023-03-29T14:06:15.448 INFO:tasks.rados.rados.0.smithi017.stdout:925: write oid 33 current snap is 103 2023-03-29T14:06:15.448 INFO:tasks.rados.rados.0.smithi017.stdout:925: seq_num 456 ranges {463992=778371,1701083=625825,2956414=658979} 2023-03-29T14:06:15.454 INFO:tasks.rados.rados.0.smithi017.stdout:925: writing smithi017107884-33 from 463992 to 1242363 tid 1 2023-03-29T14:06:15.460 INFO:tasks.rados.rados.0.smithi017.stdout:925: writing smithi017107884-33 from 1701083 to 2326908 tid 2 2023-03-29T14:06:15.467 INFO:tasks.rados.rados.0.smithi017.stdout:925: writing smithi017107884-33 from 2956414 to 3615393 tid 3 2023-03-29T14:06:15.468 INFO:tasks.rados.rados.0.smithi017.stdout:926: write oid 26 current snap is 103 2023-03-29T14:06:15.468 INFO:tasks.rados.rados.0.smithi017.stdout:926: seq_num 457 ranges {480799=569973,1475918=769483,2700190=586604} 2023-03-29T14:06:15.474 INFO:tasks.rados.rados.0.smithi017.stdout:926: writing smithi017107884-26 from 480799 to 1050772 tid 1 2023-03-29T14:06:15.480 INFO:tasks.rados.rados.0.smithi017.stdout:926: writing smithi017107884-26 from 1475918 to 2245401 tid 2 2023-03-29T14:06:15.486 INFO:tasks.rados.rados.0.smithi017.stdout:926: writing smithi017107884-26 from 2700190 to 3286794 tid 3 2023-03-29T14:06:15.487 INFO:tasks.rados.rados.0.smithi017.stdout:927: write (excl) oid 27 current snap is 103 2023-03-29T14:06:15.487 INFO:tasks.rados.rados.0.smithi017.stdout:927: seq_num 458 ranges {497606=645222,1934400=629494,3243966=453640} 2023-03-29T14:06:15.493 INFO:tasks.rados.rados.0.smithi017.stdout:927: writing smithi017107884-27 from 497606 to 1142828 tid 1 2023-03-29T14:06:15.501 INFO:tasks.rados.rados.0.smithi017.stdout:927: writing smithi017107884-27 from 1934400 to 2563894 tid 2 2023-03-29T14:06:15.507 INFO:tasks.rados.rados.0.smithi017.stdout:927: writing smithi017107884-27 from 3243966 to 3697606 tid 3 2023-03-29T14:06:15.507 INFO:tasks.rados.rados.0.smithi017.stdout:928: write (excl) oid 19 current snap is 103 2023-03-29T14:06:15.507 INFO:tasks.rados.rados.0.smithi017.stdout:926: finishing write tid 1 to smithi017107884-26 2023-03-29T14:06:15.507 INFO:tasks.rados.rados.0.smithi017.stdout:928: seq_num 459 ranges {514413=720471,1876529=773152,3438683=275730} 2023-03-29T14:06:15.514 INFO:tasks.rados.rados.0.smithi017.stdout:928: writing smithi017107884-19 from 514413 to 1234884 tid 1 2023-03-29T14:06:15.522 INFO:tasks.rados.rados.0.smithi017.stdout:928: writing smithi017107884-19 from 1876529 to 2649681 tid 2 2023-03-29T14:06:15.528 INFO:tasks.rados.rados.0.smithi017.stdout:928: writing smithi017107884-19 from 3438683 to 3714413 tid 3 2023-03-29T14:06:15.528 INFO:tasks.rados.rados.0.smithi017.stdout:929: delete oid 21 current snap is 103 2023-03-29T14:06:15.528 INFO:tasks.rados.rados.0.smithi017.stdout:926: finishing write tid 2 to smithi017107884-26 2023-03-29T14:06:15.564 INFO:tasks.rados.rados.0.smithi017.stdout:926: finishing write tid 3 to smithi017107884-26 2023-03-29T14:06:15.598 INFO:tasks.rados.rados.0.smithi017.stdout:925: finishing write tid 1 to smithi017107884-33 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:926: finishing write tid 4 to smithi017107884-26 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:926: finishing write tid 5 to smithi017107884-26 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:926: oid 26 updating version 0 to 93 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:926: oid 26 version 93 is already newer than 93 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:926: oid 26 version 93 is already newer than 93 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:926: oid 26 version 93 is already newer than 93 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 93 (ObjNum 457 snap 103 seq_num 457) dirty exists 2023-03-29T14:06:15.605 INFO:tasks.rados.rados.0.smithi017.stdout:926: left oid 26 (ObjNum 457 snap 103 seq_num 457) 2023-03-29T14:06:15.606 INFO:tasks.rados.rados.0.smithi017.stdout:925: finishing write tid 2 to smithi017107884-33 2023-03-29T14:06:15.613 INFO:tasks.rados.rados.0.smithi017.stdout:925: finishing write tid 3 to smithi017107884-33 2023-03-29T14:06:15.616 INFO:tasks.rados.rados.0.smithi017.stdout:925: finishing write tid 4 to smithi017107884-33 2023-03-29T14:06:15.622 INFO:tasks.rados.rados.0.smithi017.stdout:925: finishing write tid 5 to smithi017107884-33 2023-03-29T14:06:15.622 INFO:tasks.rados.rados.0.smithi017.stdout:925: oid 33 updating version 0 to 124 2023-03-29T14:06:15.622 INFO:tasks.rados.rados.0.smithi017.stdout:925: oid 33 version 124 is already newer than 124 2023-03-29T14:06:15.622 INFO:tasks.rados.rados.0.smithi017.stdout:925: oid 33 version 124 is already newer than 124 2023-03-29T14:06:15.622 INFO:tasks.rados.rados.0.smithi017.stdout:925: oid 33 version 124 is already newer than 124 2023-03-29T14:06:15.622 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 124 (ObjNum 456 snap 103 seq_num 456) dirty exists 2023-03-29T14:06:15.623 INFO:tasks.rados.rados.0.smithi017.stdout:925: left oid 33 (ObjNum 456 snap 103 seq_num 456) 2023-03-29T14:06:15.627 INFO:tasks.rados.rados.0.smithi017.stdout:927: finishing write tid 1 to smithi017107884-27 2023-03-29T14:06:15.671 INFO:tasks.rados.rados.0.smithi017.stdout:928: finishing write tid 1 to smithi017107884-19 2023-03-29T14:06:15.690 INFO:tasks.rados.rados.0.smithi017.stdout:927: finishing write tid 2 to smithi017107884-27 2023-03-29T14:06:15.690 INFO:tasks.rados.rados.0.smithi017.stdout:927: finishing write tid 3 to smithi017107884-27 2023-03-29T14:06:15.691 INFO:tasks.rados.rados.0.smithi017.stdout:927: finishing write tid 4 to smithi017107884-27 2023-03-29T14:06:15.693 INFO:tasks.rados.rados.0.smithi017.stdout:927: finishing write tid 5 to smithi017107884-27 2023-03-29T14:06:15.694 INFO:tasks.rados.rados.0.smithi017.stdout:927: oid 27 updating version 0 to 97 2023-03-29T14:06:15.694 INFO:tasks.rados.rados.0.smithi017.stdout:927: oid 27 version 97 is already newer than 97 2023-03-29T14:06:15.694 INFO:tasks.rados.rados.0.smithi017.stdout:927: oid 27 version 97 is already newer than 97 2023-03-29T14:06:15.694 INFO:tasks.rados.rados.0.smithi017.stdout:927: oid 27 version 97 is already newer than 94 2023-03-29T14:06:15.694 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 97 (ObjNum 458 snap 103 seq_num 458) dirty exists 2023-03-29T14:06:15.694 INFO:tasks.rados.rados.0.smithi017.stdout:927: left oid 27 (ObjNum 458 snap 103 seq_num 458) 2023-03-29T14:06:15.703 INFO:tasks.rados.rados.0.smithi017.stdout:928: finishing write tid 2 to smithi017107884-19 2023-03-29T14:06:15.705 INFO:tasks.rados.rados.0.smithi017.stdout:928: finishing write tid 3 to smithi017107884-19 2023-03-29T14:06:15.711 INFO:tasks.rados.rados.0.smithi017.stdout:928: finishing write tid 4 to smithi017107884-19 2023-03-29T14:06:15.714 INFO:tasks.rados.rados.0.smithi017.stdout:928: finishing write tid 5 to smithi017107884-19 2023-03-29T14:06:15.714 INFO:tasks.rados.rados.0.smithi017.stdout:928: oid 19 updating version 0 to 125 2023-03-29T14:06:15.714 INFO:tasks.rados.rados.0.smithi017.stdout:928: oid 19 updating version 125 to 129 2023-03-29T14:06:15.714 INFO:tasks.rados.rados.0.smithi017.stdout:928: oid 19 version 129 is already newer than 129 2023-03-29T14:06:15.714 INFO:tasks.rados.rados.0.smithi017.stdout:928: oid 19 version 129 is already newer than 129 2023-03-29T14:06:15.714 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 129 (ObjNum 459 snap 103 seq_num 459) dirty exists 2023-03-29T14:06:15.714 INFO:tasks.rados.rados.0.smithi017.stdout:928: left oid 19 (ObjNum 459 snap 103 seq_num 459) 2023-03-29T14:06:15.750 INFO:tasks.rados.rados.0.smithi017.stdout:925: done (4 left) 2023-03-29T14:06:15.750 INFO:tasks.rados.rados.0.smithi017.stdout:926: done (3 left) 2023-03-29T14:06:15.750 INFO:tasks.rados.rados.0.smithi017.stdout:927: done (2 left) 2023-03-29T14:06:15.750 INFO:tasks.rados.rados.0.smithi017.stdout:928: done (1 left) 2023-03-29T14:06:15.750 INFO:tasks.rados.rados.0.smithi017.stdout:929: done (0 left) 2023-03-29T14:06:15.750 INFO:tasks.rados.rados.0.smithi017.stdout:930: delete oid 43 current snap is 103 2023-03-29T14:06:15.780 INFO:tasks.rados.rados.0.smithi017.stdout:930: done (0 left) 2023-03-29T14:06:15.781 INFO:tasks.rados.rados.0.smithi017.stdout:931: snap_create 2023-03-29T14:06:16.022 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:16.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~66s 2023-03-29T14:06:16.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:16.151+0000 7f0d739a8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:16.155+0000 7f0d739a8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:16.220 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:16.221 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:931: done (0 left) 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:932: read oid 42 snap -1 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:932: expect (ObjNum 433 snap 99 seq_num 433) 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:933: read oid 13 snap 102 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:933: expect deleted 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:934: read oid 41 snap 103 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:934: expect deleted 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:935: write (excl) oid 48 current snap is 104 2023-03-29T14:06:16.330 INFO:tasks.rados.rados.0.smithi017.stdout:935: seq_num 462 ranges {564834=546218,1651975=520479,2720952=559317} 2023-03-29T14:06:16.336 INFO:tasks.rados.rados.0.smithi017.stdout:935: writing smithi017107884-48 from 564834 to 1111052 tid 1 2023-03-29T14:06:16.342 INFO:tasks.rados.rados.0.smithi017.stdout:935: writing smithi017107884-48 from 1651975 to 2172454 tid 2 2023-03-29T14:06:16.348 INFO:tasks.rados.rados.0.smithi017.stdout:935: writing smithi017107884-48 from 2720952 to 3280269 tid 3 2023-03-29T14:06:16.348 INFO:tasks.rados.rados.0.smithi017.stdout:936: write (excl) oid 8 current snap is 104 2023-03-29T14:06:16.348 INFO:tasks.rados.rados.0.smithi017.stdout:936: seq_num 463 ranges {581641=621467,1994104=780490,3432022=349619} 2023-03-29T14:06:16.354 INFO:tasks.rados.rados.0.smithi017.stdout:936: writing smithi017107884-8 from 581641 to 1203108 tid 1 2023-03-29T14:06:16.364 INFO:tasks.rados.rados.0.smithi017.stdout:936: writing smithi017107884-8 from 1994104 to 2774594 tid 2 2023-03-29T14:06:16.370 INFO:tasks.rados.rados.0.smithi017.stdout:936: writing smithi017107884-8 from 3432022 to 3781641 tid 3 2023-03-29T14:06:16.370 INFO:tasks.rados.rados.0.smithi017.stdout:937: snap_create 2023-03-29T14:06:16.383 INFO:tasks.rados.rados.0.smithi017.stdout:932: expect (ObjNum 433 snap 99 seq_num 433) 2023-03-29T14:06:16.604 INFO:tasks.rados.rados.0.smithi017.stdout:936: finishing write tid 1 to smithi017107884-8 2023-03-29T14:06:16.604 INFO:tasks.rados.rados.0.smithi017.stdout:935: finishing write tid 1 to smithi017107884-48 2023-03-29T14:06:16.604 INFO:tasks.rados.rados.0.smithi017.stdout:935: finishing write tid 2 to smithi017107884-48 2023-03-29T14:06:16.604 INFO:tasks.rados.rados.0.smithi017.stdout:936: finishing write tid 2 to smithi017107884-8 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:936: finishing write tid 3 to smithi017107884-8 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: finishing write tid 3 to smithi017107884-48 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: finishing write tid 4 to smithi017107884-48 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:936: finishing write tid 4 to smithi017107884-8 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: finishing write tid 5 to smithi017107884-48 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: oid 48 updating version 0 to 129 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: oid 48 version 129 is already newer than 129 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: oid 48 version 129 is already newer than 129 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: oid 48 version 129 is already newer than 126 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 129 (ObjNum 462 snap 104 seq_num 462) dirty exists 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:935: left oid 48 (ObjNum 462 snap 104 seq_num 462) 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:936: finishing write tid 5 to smithi017107884-8 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:936: oid 8 updating version 0 to 130 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:936: oid 8 updating version 130 to 134 2023-03-29T14:06:16.605 INFO:tasks.rados.rados.0.smithi017.stdout:936: oid 8 version 134 is already newer than 134 2023-03-29T14:06:16.606 INFO:tasks.rados.rados.0.smithi017.stdout:936: oid 8 version 134 is already newer than 134 2023-03-29T14:06:16.606 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 134 (ObjNum 463 snap 104 seq_num 463) dirty exists 2023-03-29T14:06:16.606 INFO:tasks.rados.rados.0.smithi017.stdout:936: left oid 8 (ObjNum 463 snap 104 seq_num 463) 2023-03-29T14:06:17.333 INFO:tasks.rados.rados.0.smithi017.stdout:932: done (5 left) 2023-03-29T14:06:17.333 INFO:tasks.rados.rados.0.smithi017.stdout:933: done (4 left) 2023-03-29T14:06:17.333 INFO:tasks.rados.rados.0.smithi017.stdout:934: done (3 left) 2023-03-29T14:06:17.334 INFO:tasks.rados.rados.0.smithi017.stdout:935: done (2 left) 2023-03-29T14:06:17.334 INFO:tasks.rados.rados.0.smithi017.stdout:936: done (1 left) 2023-03-29T14:06:17.334 INFO:tasks.rados.rados.0.smithi017.stdout:937: done (0 left) 2023-03-29T14:06:17.334 INFO:tasks.rados.rados.0.smithi017.stdout:938: snap_create 2023-03-29T14:06:18.218 INFO:tasks.rados.rados.0.smithi017.stdout:938: done (0 left) 2023-03-29T14:06:18.218 INFO:tasks.rados.rados.0.smithi017.stdout:939: write (excl) oid 44 current snap is 106 2023-03-29T14:06:18.218 INFO:tasks.rados.rados.0.smithi017.stdout:939: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2023-03-29T14:06:18.223 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:18.224 INFO:tasks.rados.rados.0.smithi017.stdout:939: writing smithi017107884-44 from 598448 to 1011517 tid 1 2023-03-29T14:06:18.231 INFO:tasks.rados.rados.0.smithi017.stdout:939: writing smithi017107884-44 from 1768939 to 2293087 tid 2 2023-03-29T14:06:18.237 INFO:tasks.rados.rados.0.smithi017.stdout:939: writing smithi017107884-44 from 2775798 to 3357659 tid 3 2023-03-29T14:06:18.240 INFO:tasks.rados.rados.0.smithi017.stdout:939: writing smithi017107884-44 from 3781488 to 3798448 tid 4 2023-03-29T14:06:18.240 INFO:tasks.rados.rados.0.smithi017.stdout:940: read oid 11 snap 104 2023-03-29T14:06:18.240 INFO:tasks.rados.rados.0.smithi017.stdout:940: expect (ObjNum 395 snap 94 seq_num 395) 2023-03-29T14:06:18.240 INFO:tasks.rados.rados.0.smithi017.stdout:941: read oid 48 snap -1 2023-03-29T14:06:18.240 INFO:tasks.rados.rados.0.smithi017.stdout:941: expect (ObjNum 462 snap 104 seq_num 462) 2023-03-29T14:06:18.240 INFO:tasks.rados.rados.0.smithi017.stdout:942: snap_remove snap 102 2023-03-29T14:06:18.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:18.362+0000 7f26986da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:18.366 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:18.366+0000 7f26986da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:18.455 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:18.458 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:18.458 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:19.061 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:19.061 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:19.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:19.193+0000 7fc30adbf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:19.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:19.199+0000 7fc30adbf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:19.222 INFO:tasks.rados.rados.0.smithi017.stdout:942: done (3 left) 2023-03-29T14:06:19.222 INFO:tasks.rados.rados.0.smithi017.stdout:943: snap_remove snap 105 2023-03-29T14:06:19.404 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:19.407 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:19.424 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2023-03-29T14:06:19.220914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189557552,"num_objects":526,"num_object_clones":430,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2241,"num_read_kb":1235417,"num_write":3004,"num_write_kb":801596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1572,"ondisk_log_size":1572,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1069932,"kb_used_data":1008344,"kb_used_omap":0,"kb_used_meta":56320,"kb_avail":376417428,"statfs":{"total":386547056640,"available":385451446272,"internally_reserved":0,"allocated":1032544256,"data_stored":1237106371,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":57671680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7495988,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":60,"num_read_kb":32783,"num_write":110,"num_write_kb":37328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.896135"},"pg_stats":[{"pgid":"3.f","version":"222'66","reported_seq":115,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:15.347844+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:15.347844+0000","last_peered":"2023-03-29T14:06:15.347844+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:15.347844+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345022,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":21195,"num_write":52,"num_write_kb":14796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"209'21","reported_seq":233,"reported_epoch":229,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:15.400412+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:15.400412+0000","last_peered":"2023-03-29T14:06:15.400412+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:15.400412+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":10,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":41,"num_read_kb":25428,"num_write":44,"num_write_kb":12992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"230'128","reported_seq":469,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-03-29T14:06:16.473546+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:16.473546+0000","last_peered":"2023-03-29T14:06:16.473546+0000","last_clean":"2023-03-29T14:06:16.473546+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:16.473546+0000","last_undegraded":"2023-03-29T14:06:16.473546+0000","last_fullsized":"2023-03-29T14:06:16.473546+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18118342,"num_objects":42,"num_object_clones":32,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":171,"num_read_kb":113922,"num_write":246,"num_write_kb":72200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"221'144","reported_seq":510,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-03-29T14:06:16.377511+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:16.377511+0000","last_peered":"2023-03-29T14:06:16.377511+0000","last_clean":"2023-03-29T14:06:16.377511+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:16.377511+0000","last_undegraded":"2023-03-29T14:06:16.377511+0000","last_fullsized":"2023-03-29T14:06:16.377511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521710,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":215,"num_read_kb":127247,"num_write":284,"num_write_kb":79564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"228'131","reported_seq":472,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-03-29T14:06:14.998364+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:14.998364+0000","last_peered":"2023-03-29T14:06:14.998364+0000","last_clean":"2023-03-29T14:06:14.998364+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:14.998364+0000","last_undegraded":"2023-03-29T14:06:14.998364+0000","last_fullsized":"2023-03-29T14:06:14.998364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252600,"num_objects":50,"num_object_clones":42,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":185,"num_read_kb":110006,"num_write":262,"num_write_kb":78504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":189,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-03-29T14:06:15.344323+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:15.344323+0000","last_peered":"2023-03-29T14:06:15.344323+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:15.344323+0000","last_undegraded":"2023-03-29T14:06:15.344323+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"228'91","reported_seq":382,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:15.343456+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:15.343456+0000","last_peered":"2023-03-29T14:06:15.343456+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:15.343456+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":112,"num_read_kb":51658,"num_write":164,"num_write_kb":48718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"228'64","reported_seq":327,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-03-29T14:06:15.346511+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:15.346511+0000","last_peered":"2023-03-29T14:06:15.346511+0000","last_clean":"2023-03-29T14:06:15.346511+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:15.346511+0000","last_undegraded":"2023-03-29T14:06:15.346511+0000","last_fullsized":"2023-03-29T14:06:15.346511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7059072,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":49142,"num_write":126,"num_write_kb":35664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":183,"reported_epoch":229,"state":"active+undersized","last_fresh":"2023-03-29T14:06:14.474064+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:14.474064+0000","last_peered":"2023-03-29T14:06:14.474064+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:14.474064+0000","last_undegraded":"2023-03-29T14:06:14.474064+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":55,"reported_epoch":229,"state":"active+undersized","last_fresh":"2023-03-29T14:06:14.444458+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:14.444458+0000","last_peered":"2023-03-29T14:06:14.444458+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:14.444458+0000","last_undegraded":"2023-03-29T14:06:14.444458+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7262,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":192,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-03-29T14:06:15.336487+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:15.336487+0000","last_peered":"2023-03-29T14:06:15.336487+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:15.336487+0000","last_undegraded":"2023-03-29T14:06:15.336487+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"222'78","reported_seq":357,"reported_epoch":229,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:14.423002+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:14.423002+0000","last_peered":"2023-03-29T14:06:14.423002+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:14.423002+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13109724,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":70994,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":229,"state":"active+undersized","last_fresh":"2023-03-29T14:06:14.440899+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:14.440899+0000","last_peered":"2023-03-29T14:06:14.440899+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:14.440899+0000","last_undegraded":"2023-03-29T14:06:14.440899+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"219'87","reported_seq":375,"reported_epoch":229,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:15.755466+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:15.755466+0000","last_peered":"2023-03-29T14:06:15.755466+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:15.755466+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6319564,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":108,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":192,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-03-29T14:06:15.349330+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:15.349330+0000","last_peered":"2023-03-29T14:06:15.349330+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:15.349330+0000","last_undegraded":"2023-03-29T14:06:15.349330+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":408,"reported_epoch":229,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:14.427332+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:14.427332+0000","last_peered":"2023-03-29T14:06:14.427332+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:14.427332+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"229'99","reported_seq":447,"reported_epoch":230,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:15.695026+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:15.695026+0000","last_peered":"2023-03-29T14:06:15.695026+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:15.695026+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21580584,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":210,"num_read_kb":131591,"num_write":198,"num_write_kb":58252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-03-29T14:06:15.339968+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:15.339968+0000","last_peered":"2023-03-29T14:06:15.339968+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:15.339968+0000","last_undegraded":"2023-03-29T14:06:15.339968+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"225'196","reported_seq":640,"reported_epoch":229,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:14.853157+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:14.853157+0000","last_peered":"2023-03-29T14:06:14.853157+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:14.853157+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261416,"num_objects":70,"num_object_clones":56,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":339,"num_read_kb":208383,"num_write":364,"num_write_kb":105456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":183,"reported_epoch":229,"state":"active+undersized","last_fresh":"2023-03-29T14:06:14.448423+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:14.448423+0000","last_peered":"2023-03-29T14:06:14.448423+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:14.448423+0000","last_undegraded":"2023-03-29T14:06:14.448423+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"202'63","reported_seq":328,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-03-29T14:06:14.472178+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:14.472178+0000","last_peered":"2023-03-29T14:06:14.472178+0000","last_clean":"2023-03-29T14:06:14.472178+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:14.472178+0000","last_undegraded":"2023-03-29T14:06:14.472178+0000","last_fullsized":"2023-03-29T14:06:14.472178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8639062,"num_objects":22,"num_object_clones":18,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":59594,"num_write":134,"num_write_kb":37726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"214'136","reported_seq":474,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-03-29T14:06:14.432606+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:14.432606+0000","last_peered":"2023-03-29T14:06:14.432606+0000","last_clean":"2023-03-29T14:06:14.432606+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:14.432606+0000","last_undegraded":"2023-03-29T14:06:14.432606+0000","last_fullsized":"2023-03-29T14:06:14.432606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723130,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":173,"num_read_kb":98003,"num_write":260,"num_write_kb":76742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":183,"reported_epoch":229,"state":"active+undersized","last_fresh":"2023-03-29T14:06:14.459863+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:14.459863+0000","last_peered":"2023-03-29T14:06:14.459863+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:14.459863+0000","last_undegraded":"2023-03-29T14:06:14.459863+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":192,"reported_epoch":230,"state":"active+undersized","last_fresh":"2023-03-29T14:06:15.333876+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:15.333876+0000","last_peered":"2023-03-29T14:06:15.333876+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:15.333876+0000","last_undegraded":"2023-03-29T14:06:15.333876+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"229'129","reported_seq":473,"reported_epoch":229,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:15.696169+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:15.696169+0000","last_peered":"2023-03-29T14:06:15.696169+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:15.696169+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21748654,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":185,"num_read_kb":109999,"num_write":254,"num_write_kb":76504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":188376816,"num_objects":522,"num_object_clones":430,"num_object_copies":1566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2099,"num_read_kb":1235297,"num_write":2730,"num_write_kb":797808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1475,"ondisk_log_size":1475,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":290900,"kb_used_data":274460,"kb_used_omap":0,"kb_used_meta":14144,"kb_avail":94080940,"statfs":{"total":96636764160,"available":96338882560,"internally_reserved":0,"allocated":281047040,"data_stored":332382410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14483456},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":322736,"kb_used_data":306836,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94049104,"statfs":{"total":96636764160,"available":96306282496,"internally_reserved":0,"allocated":314200064,"data_stored":379505038,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":300352,"kb_used_data":283208,"kb_used_omap":0,"kb_used_meta":14848,"kb_avail":94071488,"statfs":{"total":96636764160,"available":96329203712,"internally_reserved":0,"allocated":290004992,"data_stored":351986698,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15204352},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:19.428 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.3 1.0 3.2 3.1 3.6 2023-03-29T14:06:19.565 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:19.555+0000 7fafab942700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:19.569 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:19.569+0000 7fafab942700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3,3.6] on osd.0 to force-recovery; instructing pg(s) [1.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.2,3.f] on osd.3 to force-recovery; 2023-03-29T14:06:19.773 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:06:20.224 INFO:tasks.rados.rados.0.smithi017.stdout:943: done (3 left) 2023-03-29T14:06:20.224 INFO:tasks.rados.rados.0.smithi017.stdout:944: write oid 19 current snap is 106 2023-03-29T14:06:20.224 INFO:tasks.rados.rados.0.smithi017.stdout:944: seq_num 465 ranges {615255=488318,1827421=784159,3203221=612034} 2023-03-29T14:06:20.229 INFO:tasks.rados.rados.0.smithi017.stdout:944: writing smithi017107884-19 from 615255 to 1103573 tid 1 2023-03-29T14:06:20.238 INFO:tasks.rados.rados.0.smithi017.stdout:944: writing smithi017107884-19 from 1827421 to 2611580 tid 2 2023-03-29T14:06:20.244 INFO:tasks.rados.rados.0.smithi017.stdout:944: writing smithi017107884-19 from 3203221 to 3815255 tid 3 2023-03-29T14:06:20.245 INFO:tasks.rados.rados.0.smithi017.stdout:945: snap_remove snap 103 2023-03-29T14:06:20.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:20.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:20.589+0000 7fba364fd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:20.593 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:20.593+0000 7fba364fd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:20.654 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:20.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:20.659 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:21.227 INFO:tasks.rados.rados.0.smithi017.stdout:945: done (4 left) 2023-03-29T14:06:21.227 INFO:tasks.rados.rados.0.smithi017.stdout:946: snap_create 2023-03-29T14:06:21.227 INFO:tasks.rados.rados.0.smithi017.stdout:941: expect (ObjNum 462 snap 104 seq_num 462) 2023-03-29T14:06:21.374 INFO:tasks.rados.rados.0.smithi017.stdout:940: expect (ObjNum 395 snap 94 seq_num 395) 2023-03-29T14:06:21.478 INFO:tasks.rados.rados.0.smithi017.stdout:939: finishing write tid 1 to smithi017107884-44 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: finishing write tid 2 to smithi017107884-44 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: finishing write tid 3 to smithi017107884-44 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: finishing write tid 4 to smithi017107884-44 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: finishing write tid 5 to smithi017107884-44 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: finishing write tid 6 to smithi017107884-44 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: oid 44 updating version 0 to 69 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: oid 44 version 69 is already newer than 69 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: oid 44 version 69 is already newer than 69 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: oid 44 version 69 is already newer than 64 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: oid 44 version 69 is already newer than 69 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 69 (ObjNum 464 snap 106 seq_num 464) dirty exists 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:939: left oid 44 (ObjNum 464 snap 106 seq_num 464) 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:944: finishing write tid 1 to smithi017107884-19 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:944: finishing write tid 2 to smithi017107884-19 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:944: finishing write tid 3 to smithi017107884-19 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:944: finishing write tid 4 to smithi017107884-19 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:944: finishing write tid 5 to smithi017107884-19 2023-03-29T14:06:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:944: oid 19 updating version 0 to 139 2023-03-29T14:06:21.480 INFO:tasks.rados.rados.0.smithi017.stdout:944: oid 19 version 139 is already newer than 139 2023-03-29T14:06:21.480 INFO:tasks.rados.rados.0.smithi017.stdout:944: oid 19 version 139 is already newer than 139 2023-03-29T14:06:21.480 INFO:tasks.rados.rados.0.smithi017.stdout:944: oid 19 version 139 is already newer than 139 2023-03-29T14:06:21.480 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 139 (ObjNum 465 snap 106 seq_num 465) dirty exists 2023-03-29T14:06:21.480 INFO:tasks.rados.rados.0.smithi017.stdout:944: left oid 19 (ObjNum 465 snap 106 seq_num 465) 2023-03-29T14:06:21.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~72s 2023-03-29T14:06:22.231 INFO:tasks.rados.rados.0.smithi017.stdout:939: done (4 left) 2023-03-29T14:06:22.231 INFO:tasks.rados.rados.0.smithi017.stdout:940: done (3 left) 2023-03-29T14:06:22.231 INFO:tasks.rados.rados.0.smithi017.stdout:941: done (2 left) 2023-03-29T14:06:22.231 INFO:tasks.rados.rados.0.smithi017.stdout:944: done (1 left) 2023-03-29T14:06:22.231 INFO:tasks.rados.rados.0.smithi017.stdout:946: done (0 left) 2023-03-29T14:06:22.231 INFO:tasks.rados.rados.0.smithi017.stdout:947: snap_remove snap 104 2023-03-29T14:06:22.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:22.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:22.787+0000 7f9cec3c0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:22.792+0000 7f9cec3c0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:22.868 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:22.871 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:22.872 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:23.233 INFO:tasks.rados.rados.0.smithi017.stdout:947: done (0 left) 2023-03-29T14:06:23.233 INFO:tasks.rados.rados.0.smithi017.stdout:948: delete oid 19 current snap is 107 2023-03-29T14:06:23.393 INFO:tasks.rados.rados.0.smithi017.stdout:948: done (0 left) 2023-03-29T14:06:23.393 INFO:tasks.rados.rados.0.smithi017.stdout:949: write oid 48 current snap is 107 2023-03-29T14:06:23.393 INFO:tasks.rados.rados.0.smithi017.stdout:949: seq_num 467 ranges {648869=638816,1828032=787828,3141714=532030} 2023-03-29T14:06:23.400 INFO:tasks.rados.rados.0.smithi017.stdout:949: writing smithi017107884-48 from 648869 to 1287685 tid 1 2023-03-29T14:06:23.407 INFO:tasks.rados.rados.0.smithi017.stdout:949: writing smithi017107884-48 from 1828032 to 2615860 tid 2 2023-03-29T14:06:23.414 INFO:tasks.rados.rados.0.smithi017.stdout:949: writing smithi017107884-48 from 3141714 to 3673744 tid 3 2023-03-29T14:06:23.414 INFO:tasks.rados.rados.0.smithi017.stdout:950: delete oid 50 current snap is 107 2023-03-29T14:06:23.428 INFO:tasks.rados.rados.0.smithi017.stdout:949: finishing write tid 1 to smithi017107884-48 2023-03-29T14:06:23.510 INFO:tasks.rados.rados.0.smithi017.stdout:949: finishing write tid 2 to smithi017107884-48 2023-03-29T14:06:23.553 INFO:tasks.rados.rados.0.smithi017.stdout:950: done (1 left) 2023-03-29T14:06:23.554 INFO:tasks.rados.rados.0.smithi017.stdout:951: snap_remove snap 106 2023-03-29T14:06:24.239 INFO:tasks.rados.rados.0.smithi017.stdout:951: done (1 left) 2023-03-29T14:06:24.239 INFO:tasks.rados.rados.0.smithi017.stdout:952: write oid 47 current snap is 107 2023-03-29T14:06:24.239 INFO:tasks.rados.rados.0.smithi017.stdout:949: finishing write tid 3 to smithi017107884-48 2023-03-29T14:06:24.239 INFO:tasks.rados.rados.0.smithi017.stdout:952: seq_num 469 ranges {682483=789314,1944996=791497,3196560=554574} 2023-03-29T14:06:24.247 INFO:tasks.rados.rados.0.smithi017.stdout:952: writing smithi017107884-47 from 682483 to 1471797 tid 1 2023-03-29T14:06:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:952: writing smithi017107884-47 from 1944996 to 2736493 tid 2 2023-03-29T14:06:24.260 INFO:tasks.rados.rados.0.smithi017.stdout:952: writing smithi017107884-47 from 3196560 to 3751134 tid 3 2023-03-29T14:06:24.260 INFO:tasks.rados.rados.0.smithi017.stdout:953: write (excl) oid 38 current snap is 107 2023-03-29T14:06:24.260 INFO:tasks.rados.rados.0.smithi017.stdout:949: finishing write tid 4 to smithi017107884-48 2023-03-29T14:06:24.260 INFO:tasks.rados.rados.0.smithi017.stdout:953: seq_num 470 ranges {699290=464563,1603478=535155,2823983=765846} 2023-03-29T14:06:24.266 INFO:tasks.rados.rados.0.smithi017.stdout:953: writing smithi017107884-38 from 699290 to 1163853 tid 1 2023-03-29T14:06:24.272 INFO:tasks.rados.rados.0.smithi017.stdout:953: writing smithi017107884-38 from 1603478 to 2138633 tid 2 2023-03-29T14:06:24.280 INFO:tasks.rados.rados.0.smithi017.stdout:953: writing smithi017107884-38 from 2823983 to 3589829 tid 3 2023-03-29T14:06:24.280 INFO:tasks.rados.rados.0.smithi017.stdout:954: read oid 31 snap -1 2023-03-29T14:06:24.280 INFO:tasks.rados.rados.0.smithi017.stdout:954: expect deleted 2023-03-29T14:06:24.280 INFO:tasks.rados.rados.0.smithi017.stdout:955: read oid 4 snap -1 2023-03-29T14:06:24.280 INFO:tasks.rados.rados.0.smithi017.stdout:955: expect (ObjNum 419 snap 97 seq_num 419) 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:956: read oid 1 snap -1 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:956: expect deleted 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:957: delete oid 11 current snap is 107 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:949: finishing write tid 5 to smithi017107884-48 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:949: oid 48 updating version 0 to 134 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:949: oid 48 version 134 is already newer than 134 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:949: oid 48 version 134 is already newer than 134 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:949: oid 48 version 134 is already newer than 134 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 134 (ObjNum 467 snap 107 seq_num 467) dirty exists 2023-03-29T14:06:24.281 INFO:tasks.rados.rados.0.smithi017.stdout:949: left oid 48 (ObjNum 467 snap 107 seq_num 467) 2023-03-29T14:06:24.297 INFO:tasks.rados.rados.0.smithi017.stdout:955: expect (ObjNum 419 snap 97 seq_num 419) 2023-03-29T14:06:24.421 INFO:tasks.rados.rados.0.smithi017.stdout:952: finishing write tid 1 to smithi017107884-47 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: finishing write tid 2 to smithi017107884-47 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: finishing write tid 3 to smithi017107884-47 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: finishing write tid 4 to smithi017107884-47 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: finishing write tid 5 to smithi017107884-47 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: oid 47 updating version 0 to 142 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: oid 47 version 142 is already newer than 142 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: oid 47 version 142 is already newer than 142 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: oid 47 version 142 is already newer than 142 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 142 (ObjNum 469 snap 107 seq_num 469) dirty exists 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: left oid 47 (ObjNum 469 snap 107 seq_num 469) 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:949: done (6 left) 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:952: done (5 left) 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:954: done (4 left) 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:955: done (3 left) 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:956: done (2 left) 2023-03-29T14:06:24.422 INFO:tasks.rados.rados.0.smithi017.stdout:957: done (1 left) 2023-03-29T14:06:24.423 INFO:tasks.rados.rados.0.smithi017.stdout:958: snap_create 2023-03-29T14:06:24.451 INFO:tasks.rados.rados.0.smithi017.stdout:953: finishing write tid 1 to smithi017107884-38 2023-03-29T14:06:24.476 INFO:tasks.rados.rados.0.smithi017.stdout:953: finishing write tid 2 to smithi017107884-38 2023-03-29T14:06:24.516 INFO:tasks.rados.rados.0.smithi017.stdout:953: finishing write tid 3 to smithi017107884-38 2023-03-29T14:06:24.518 INFO:tasks.rados.rados.0.smithi017.stdout:953: finishing write tid 4 to smithi017107884-38 2023-03-29T14:06:24.520 INFO:tasks.rados.rados.0.smithi017.stdout:953: finishing write tid 5 to smithi017107884-38 2023-03-29T14:06:24.520 INFO:tasks.rados.rados.0.smithi017.stdout:953: oid 38 updating version 0 to 24 2023-03-29T14:06:24.520 INFO:tasks.rados.rados.0.smithi017.stdout:953: oid 38 updating version 24 to 27 2023-03-29T14:06:24.521 INFO:tasks.rados.rados.0.smithi017.stdout:953: oid 38 version 27 is already newer than 27 2023-03-29T14:06:24.521 INFO:tasks.rados.rados.0.smithi017.stdout:953: oid 38 version 27 is already newer than 27 2023-03-29T14:06:24.521 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 27 (ObjNum 470 snap 107 seq_num 470) dirty exists 2023-03-29T14:06:24.521 INFO:tasks.rados.rados.0.smithi017.stdout:953: left oid 38 (ObjNum 470 snap 107 seq_num 470) 2023-03-29T14:06:24.785 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:24.785 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:24.786 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:24.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:24.916+0000 7f3d09e2b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:24.923+0000 7f3d09e2b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:25.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:25.010+0000 7f6af181a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:25.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:25.014+0000 7f6af181a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:25.085 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:25.088 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:25.088 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:25.145 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:25.157 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2023-03-29T14:06:24.238346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":193759302,"num_objects":532,"num_object_clones":436,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2263,"num_read_kb":1250555,"num_write":3036,"num_write_kb":811706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1593,"ondisk_log_size":1593,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1092176,"kb_used_data":1035440,"kb_used_omap":0,"kb_used_meta":56704,"kb_avail":376395184,"statfs":{"total":386547056640,"available":385428668416,"internally_reserved":0,"allocated":1060290560,"data_stored":1268939057,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":58064896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4201750,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":15138,"num_write":32,"num_write_kb":10110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016083"},"pg_stats":[{"pgid":"3.f","version":"222'66","reported_seq":118,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:18.268894+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:18.268894+0000","last_peered":"2023-03-29T14:06:18.268894+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:18.268894+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345022,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":21195,"num_write":52,"num_write_kb":14796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"229'23","reported_seq":236,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:17.606584+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:17.606584+0000","last_peered":"2023-03-29T14:06:17.606584+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:17.606584+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":10,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":41,"num_read_kb":25428,"num_write":44,"num_write_kb":12992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"230'129","reported_seq":476,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-03-29T14:06:18.283627+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:18.283627+0000","last_peered":"2023-03-29T14:06:18.283627+0000","last_clean":"2023-03-29T14:06:18.283627+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:18.283627+0000","last_undegraded":"2023-03-29T14:06:18.283627+0000","last_fullsized":"2023-03-29T14:06:18.283627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18118342,"num_objects":42,"num_object_clones":32,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":179,"num_read_kb":122865,"num_write":246,"num_write_kb":72200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"221'144","reported_seq":513,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-03-29T14:06:18.267045+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:18.267045+0000","last_peered":"2023-03-29T14:06:18.267045+0000","last_clean":"2023-03-29T14:06:18.267045+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:18.267045+0000","last_undegraded":"2023-03-29T14:06:18.267045+0000","last_fullsized":"2023-03-29T14:06:18.267045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521710,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":215,"num_read_kb":127247,"num_write":284,"num_write_kb":79564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"228'131","reported_seq":476,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-03-29T14:06:18.293682+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:18.293682+0000","last_peered":"2023-03-29T14:06:18.293682+0000","last_clean":"2023-03-29T14:06:18.293682+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:18.293682+0000","last_undegraded":"2023-03-29T14:06:18.293682+0000","last_fullsized":"2023-03-29T14:06:18.293682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252600,"num_objects":50,"num_object_clones":42,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":185,"num_read_kb":110006,"num_write":262,"num_write_kb":78504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":192,"reported_epoch":234,"state":"active+undersized","last_fresh":"2023-03-29T14:06:18.261505+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:18.261505+0000","last_peered":"2023-03-29T14:06:18.261505+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:18.261505+0000","last_undegraded":"2023-03-29T14:06:18.261505+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"228'91","reported_seq":385,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:18.264807+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:18.264807+0000","last_peered":"2023-03-29T14:06:18.264807+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:18.264807+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":112,"num_read_kb":51658,"num_write":164,"num_write_kb":48718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"228'64","reported_seq":330,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-03-29T14:06:18.253111+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:18.253111+0000","last_peered":"2023-03-29T14:06:18.253111+0000","last_clean":"2023-03-29T14:06:18.253111+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:18.253111+0000","last_undegraded":"2023-03-29T14:06:18.253111+0000","last_fullsized":"2023-03-29T14:06:18.253111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7059072,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":49142,"num_write":126,"num_write_kb":35664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":186,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-03-29T14:06:17.609625+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:17.609625+0000","last_peered":"2023-03-29T14:06:17.609625+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:17.609625+0000","last_undegraded":"2023-03-29T14:06:17.609625+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":60,"reported_epoch":234,"state":"active+undersized","last_fresh":"2023-03-29T14:06:19.266437+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:19.266437+0000","last_peered":"2023-03-29T14:06:19.266437+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:19.266437+0000","last_undegraded":"2023-03-29T14:06:19.266437+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7262,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":195,"reported_epoch":234,"state":"active+undersized","last_fresh":"2023-03-29T14:06:18.267774+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:18.267774+0000","last_peered":"2023-03-29T14:06:18.267774+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:18.267774+0000","last_undegraded":"2023-03-29T14:06:18.267774+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"222'78","reported_seq":362,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:19.276278+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:19.276278+0000","last_peered":"2023-03-29T14:06:19.276278+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:19.276278+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13109724,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":70994,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-03-29T14:06:17.600593+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:17.600593+0000","last_peered":"2023-03-29T14:06:17.600593+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:17.600593+0000","last_undegraded":"2023-03-29T14:06:17.600593+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"229'89","reported_seq":378,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:17.611519+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:17.611519+0000","last_peered":"2023-03-29T14:06:17.611519+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:17.611519+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6319564,"num_objects":34,"num_object_clones":28,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":108,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":195,"reported_epoch":234,"state":"active+undersized","last_fresh":"2023-03-29T14:06:18.257556+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:18.257556+0000","last_peered":"2023-03-29T14:06:18.257556+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:18.257556+0000","last_undegraded":"2023-03-29T14:06:18.257556+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":411,"reported_epoch":235,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:17.587505+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:17.587505+0000","last_peered":"2023-03-29T14:06:17.587505+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:17.587505+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"229'99","reported_seq":450,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:18.249092+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:18.249092+0000","last_peered":"2023-03-29T14:06:18.249092+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:18.249092+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21580584,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":210,"num_read_kb":131591,"num_write":198,"num_write_kb":58252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":195,"reported_epoch":234,"state":"active+undersized","last_fresh":"2023-03-29T14:06:18.254642+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:18.254642+0000","last_peered":"2023-03-29T14:06:18.254642+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:18.254642+0000","last_undegraded":"2023-03-29T14:06:18.254642+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"225'196","reported_seq":648,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:19.258636+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:19.258636+0000","last_peered":"2023-03-29T14:06:19.258636+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:19.258636+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261416,"num_objects":70,"num_object_clones":56,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":346,"num_read_kb":214575,"num_write":364,"num_write_kb":105456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":186,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-03-29T14:06:17.619385+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:17.619385+0000","last_peered":"2023-03-29T14:06:17.619385+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:17.619385+0000","last_undegraded":"2023-03-29T14:06:17.619385+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"232'69","reported_seq":342,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-03-29T14:06:18.395493+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:18.395493+0000","last_peered":"2023-03-29T14:06:18.395493+0000","last_clean":"2023-03-29T14:06:18.395493+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:18.395493+0000","last_undegraded":"2023-03-29T14:06:18.395493+0000","last_fullsized":"2023-03-29T14:06:18.395493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12067690,"num_objects":24,"num_object_clones":20,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":59595,"num_write":146,"num_write_kb":40730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"214'136","reported_seq":477,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-03-29T14:06:17.621609+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:17.621609+0000","last_peered":"2023-03-29T14:06:17.621609+0000","last_clean":"2023-03-29T14:06:17.621609+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:17.621609+0000","last_undegraded":"2023-03-29T14:06:17.621609+0000","last_fullsized":"2023-03-29T14:06:17.621609+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723130,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":173,"num_read_kb":98003,"num_write":260,"num_write_kb":76742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":235,"state":"active+undersized","last_fresh":"2023-03-29T14:06:17.585578+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:17.585578+0000","last_peered":"2023-03-29T14:06:17.585578+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:17.585578+0000","last_undegraded":"2023-03-29T14:06:17.585578+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":195,"reported_epoch":234,"state":"active+undersized","last_fresh":"2023-03-29T14:06:18.266255+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:18.266255+0000","last_peered":"2023-03-29T14:06:18.266255+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:18.266255+0000","last_undegraded":"2023-03-29T14:06:18.266255+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"234'139","reported_seq":496,"reported_epoch":234,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:20.347124+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:20.347124+0000","last_peered":"2023-03-29T14:06:20.347124+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:20.347124+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22521776,"num_objects":50,"num_object_clones":42,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":189,"num_read_kb":110001,"num_write":274,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":50}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":192578566,"num_objects":528,"num_object_clones":436,"num_object_copies":1584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2121,"num_read_kb":1250435,"num_write":2762,"num_write_kb":807918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1496,"ondisk_log_size":1496,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":291808,"kb_used_data":277592,"kb_used_omap":0,"kb_used_meta":14208,"kb_avail":94080032,"statfs":{"total":96636764160,"available":96337952768,"internally_reserved":0,"allocated":284254208,"data_stored":336187215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14548992},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":332676,"kb_used_data":317308,"kb_used_omap":0,"kb_used_meta":15360,"kb_avail":94039164,"statfs":{"total":96636764160,"available":96296103936,"internally_reserved":0,"allocated":324923392,"data_stored":392220314,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15728640},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":311748,"kb_used_data":296700,"kb_used_omap":0,"kb_used_meta":15040,"kb_avail":94060092,"statfs":{"total":96636764160,"available":96317534208,"internally_reserved":0,"allocated":303820800,"data_stored":367299303,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15400960},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:25.242 INFO:tasks.rados.rados.0.smithi017.stdout:953: done (1 left) 2023-03-29T14:06:25.243 INFO:tasks.rados.rados.0.smithi017.stdout:958: done (0 left) 2023-03-29T14:06:25.243 INFO:tasks.rados.rados.0.smithi017.stdout:959: snap_create 2023-03-29T14:06:26.244 INFO:tasks.rados.rados.0.smithi017.stdout:959: done (0 left) 2023-03-29T14:06:26.244 INFO:tasks.rados.rados.0.smithi017.stdout:960: write (excl) oid 44 current snap is 109 2023-03-29T14:06:26.245 INFO:tasks.rados.rados.0.smithi017.stdout:960: seq_num 472 ranges {732904=731414,2120442=538824,3278829=504743} 2023-03-29T14:06:26.252 INFO:tasks.rados.rados.0.smithi017.stdout:960: writing smithi017107884-44 from 732904 to 1464318 tid 1 2023-03-29T14:06:26.259 INFO:tasks.rados.rados.0.smithi017.stdout:960: writing smithi017107884-44 from 2120442 to 2659266 tid 2 2023-03-29T14:06:26.265 INFO:tasks.rados.rados.0.smithi017.stdout:960: writing smithi017107884-44 from 3278829 to 3783572 tid 3 2023-03-29T14:06:26.265 INFO:tasks.rados.rados.0.smithi017.stdout:961: delete oid 38 current snap is 109 2023-03-29T14:06:26.393 INFO:tasks.rados.rados.0.smithi017.stdout:960: finishing write tid 1 to smithi017107884-44 2023-03-29T14:06:26.420 INFO:tasks.rados.rados.0.smithi017.stdout:961: done (1 left) 2023-03-29T14:06:26.421 INFO:tasks.rados.rados.0.smithi017.stdout:962: write (excl) oid 13 current snap is 109 2023-03-29T14:06:26.421 INFO:tasks.rados.rados.0.smithi017.stdout:962: seq_num 474 ranges {766518=481912,1837406=542493,2933675=527287,3876584=89934} 2023-03-29T14:06:26.427 INFO:tasks.rados.rados.0.smithi017.stdout:962: writing smithi017107884-13 from 766518 to 1248430 tid 1 2023-03-29T14:06:26.433 INFO:tasks.rados.rados.0.smithi017.stdout:962: writing smithi017107884-13 from 1837406 to 2379899 tid 2 2023-03-29T14:06:26.439 INFO:tasks.rados.rados.0.smithi017.stdout:962: writing smithi017107884-13 from 2933675 to 3460962 tid 3 2023-03-29T14:06:26.442 INFO:tasks.rados.rados.0.smithi017.stdout:962: writing smithi017107884-13 from 3876584 to 3966518 tid 4 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:963: delete oid 17 current snap is 109 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: finishing write tid 2 to smithi017107884-44 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: finishing write tid 3 to smithi017107884-44 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: finishing write tid 4 to smithi017107884-44 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: finishing write tid 5 to smithi017107884-44 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: oid 44 updating version 0 to 70 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: oid 44 updating version 70 to 74 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: oid 44 version 74 is already newer than 74 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: oid 44 version 74 is already newer than 74 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 74 (ObjNum 472 snap 109 seq_num 472) dirty exists 2023-03-29T14:06:26.443 INFO:tasks.rados.rados.0.smithi017.stdout:960: left oid 44 (ObjNum 472 snap 109 seq_num 472) 2023-03-29T14:06:26.444 INFO:tasks.rados.rados.0.smithi017.stdout:960: done (2 left) 2023-03-29T14:06:26.444 INFO:tasks.rados.rados.0.smithi017.stdout:963: done (1 left) 2023-03-29T14:06:26.444 INFO:tasks.rados.rados.0.smithi017.stdout:964: write oid 3 current snap is 109 2023-03-29T14:06:26.444 INFO:tasks.rados.rados.0.smithi017.stdout:964: seq_num 476 ranges {400132=632410,1438017=546162} 2023-03-29T14:06:26.450 INFO:tasks.rados.rados.0.smithi017.stdout:964: writing smithi017107884-3 from 400132 to 1032542 tid 1 2023-03-29T14:06:26.456 INFO:tasks.rados.rados.0.smithi017.stdout:964: writing smithi017107884-3 from 1438017 to 1984179 tid 2 2023-03-29T14:06:26.456 INFO:tasks.rados.rados.0.smithi017.stdout:965: read oid 9 snap -1 2023-03-29T14:06:26.456 INFO:tasks.rados.rados.0.smithi017.stdout:965: expect (ObjNum 425 snap 98 seq_num 425) 2023-03-29T14:06:26.456 INFO:tasks.rados.rados.0.smithi017.stdout:966: write (excl) oid 10 current snap is 109 2023-03-29T14:06:26.456 INFO:tasks.rados.rados.0.smithi017.stdout:966: seq_num 477 ranges {416939=707659,1896499=120440} 2023-03-29T14:06:26.462 INFO:tasks.rados.rados.0.smithi017.stdout:966: writing smithi017107884-10 from 416939 to 1124598 tid 1 2023-03-29T14:06:26.467 INFO:tasks.rados.rados.0.smithi017.stdout:966: writing smithi017107884-10 from 1896499 to 2016939 tid 2 2023-03-29T14:06:26.468 INFO:tasks.rados.rados.0.smithi017.stdout:967: write oid 21 current snap is 109 2023-03-29T14:06:26.468 INFO:tasks.rados.rados.0.smithi017.stdout:967: seq_num 478 ranges {433746=782908,1954981=78765} 2023-03-29T14:06:26.475 INFO:tasks.rados.rados.0.smithi017.stdout:967: writing smithi017107884-21 from 433746 to 1216654 tid 1 2023-03-29T14:06:26.479 INFO:tasks.rados.rados.0.smithi017.stdout:967: writing smithi017107884-21 from 1954981 to 2033746 tid 2 2023-03-29T14:06:26.479 INFO:tasks.rados.rados.0.smithi017.stdout:968: snap_create 2023-03-29T14:06:26.484 INFO:tasks.rados.rados.0.smithi017.stdout:965: expect (ObjNum 425 snap 98 seq_num 425) 2023-03-29T14:06:26.800 INFO:tasks.rados.rados.0.smithi017.stdout:962: finishing write tid 1 to smithi017107884-13 2023-03-29T14:06:26.800 INFO:tasks.rados.rados.0.smithi017.stdout:967: finishing write tid 1 to smithi017107884-21 2023-03-29T14:06:26.800 INFO:tasks.rados.rados.0.smithi017.stdout:962: finishing write tid 2 to smithi017107884-13 2023-03-29T14:06:26.800 INFO:tasks.rados.rados.0.smithi017.stdout:962: finishing write tid 3 to smithi017107884-13 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:967: finishing write tid 2 to smithi017107884-21 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:967: finishing write tid 3 to smithi017107884-21 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:967: finishing write tid 4 to smithi017107884-21 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:967: oid 21 updating version 0 to 102 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:967: oid 21 version 102 is already newer than 102 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:967: oid 21 version 102 is already newer than 102 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 102 (ObjNum 478 snap 109 seq_num 478) dirty exists 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:967: left oid 21 (ObjNum 478 snap 109 seq_num 478) 2023-03-29T14:06:26.801 INFO:tasks.rados.rados.0.smithi017.stdout:962: finishing write tid 4 to smithi017107884-13 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: finishing write tid 5 to smithi017107884-13 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: finishing write tid 6 to smithi017107884-13 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: oid 13 updating version 0 to 147 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: oid 13 version 147 is already newer than 147 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: oid 13 version 147 is already newer than 147 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: oid 13 version 147 is already newer than 147 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: oid 13 version 147 is already newer than 143 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 147 (ObjNum 474 snap 109 seq_num 474) dirty exists 2023-03-29T14:06:26.802 INFO:tasks.rados.rados.0.smithi017.stdout:962: left oid 13 (ObjNum 474 snap 109 seq_num 474) 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: finishing write tid 1 to smithi017107884-3 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: finishing write tid 2 to smithi017107884-3 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:966: finishing write tid 1 to smithi017107884-10 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: finishing write tid 3 to smithi017107884-3 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: finishing write tid 4 to smithi017107884-3 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: oid 3 updating version 0 to 201 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: oid 3 version 201 is already newer than 201 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: oid 3 version 201 is already newer than 201 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 201 (ObjNum 476 snap 109 seq_num 476) dirty exists 2023-03-29T14:06:26.803 INFO:tasks.rados.rados.0.smithi017.stdout:964: left oid 3 (ObjNum 476 snap 109 seq_num 476) 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:966: finishing write tid 2 to smithi017107884-10 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:966: finishing write tid 3 to smithi017107884-10 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:966: finishing write tid 4 to smithi017107884-10 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:966: oid 10 updating version 0 to 136 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:966: oid 10 updating version 136 to 139 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:966: oid 10 version 139 is already newer than 139 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 139 (ObjNum 477 snap 109 seq_num 477) dirty exists 2023-03-29T14:06:26.804 INFO:tasks.rados.rados.0.smithi017.stdout:966: left oid 10 (ObjNum 477 snap 109 seq_num 477) 2023-03-29T14:06:27.089 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~77s 2023-03-29T14:06:27.218 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:27.217+0000 7effd8e3c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:27.221 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:27.221+0000 7effd8e3c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:962: done (5 left) 2023-03-29T14:06:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:964: done (4 left) 2023-03-29T14:06:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:965: done (3 left) 2023-03-29T14:06:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:966: done (2 left) 2023-03-29T14:06:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:967: done (1 left) 2023-03-29T14:06:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:968: done (0 left) 2023-03-29T14:06:27.251 INFO:tasks.rados.rados.0.smithi017.stdout:969: write oid 46 current snap is 110 2023-03-29T14:06:27.252 INFO:tasks.rados.rados.0.smithi017.stdout:969: seq_num 479 ranges {450553=574510,1613463=437090} 2023-03-29T14:06:27.256 INFO:tasks.rados.rados.0.smithi017.stdout:969: writing smithi017107884-46 from 450553 to 1025063 tid 1 2023-03-29T14:06:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:969: writing smithi017107884-46 from 1613463 to 2050553 tid 2 2023-03-29T14:06:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:970: read oid 45 snap 108 2023-03-29T14:06:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:970: expect deleted 2023-03-29T14:06:27.263 INFO:tasks.rados.rados.0.smithi017.stdout:971: snap_create 2023-03-29T14:06:27.288 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:27.291 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:27.291 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:27.408 INFO:tasks.rados.rados.0.smithi017.stdout:969: finishing write tid 1 to smithi017107884-46 2023-03-29T14:06:27.412 INFO:tasks.rados.rados.0.smithi017.stdout:969: finishing write tid 2 to smithi017107884-46 2023-03-29T14:06:27.415 INFO:tasks.rados.rados.0.smithi017.stdout:969: finishing write tid 3 to smithi017107884-46 2023-03-29T14:06:27.418 INFO:tasks.rados.rados.0.smithi017.stdout:969: finishing write tid 4 to smithi017107884-46 2023-03-29T14:06:27.418 INFO:tasks.rados.rados.0.smithi017.stdout:969: oid 46 updating version 0 to 135 2023-03-29T14:06:27.418 INFO:tasks.rados.rados.0.smithi017.stdout:969: oid 46 version 135 is already newer than 135 2023-03-29T14:06:27.418 INFO:tasks.rados.rados.0.smithi017.stdout:969: oid 46 version 135 is already newer than 135 2023-03-29T14:06:27.418 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 135 (ObjNum 479 snap 110 seq_num 479) dirty exists 2023-03-29T14:06:27.418 INFO:tasks.rados.rados.0.smithi017.stdout:969: left oid 46 (ObjNum 479 snap 110 seq_num 479) 2023-03-29T14:06:28.221 INFO:tasks.rados.rados.0.smithi017.stdout:969: done (2 left) 2023-03-29T14:06:28.221 INFO:tasks.rados.rados.0.smithi017.stdout:970: done (1 left) 2023-03-29T14:06:28.221 INFO:tasks.rados.rados.0.smithi017.stdout:971: done (0 left) 2023-03-29T14:06:28.222 INFO:tasks.rados.rados.0.smithi017.stdout:972: read oid 26 snap 108 2023-03-29T14:06:28.222 INFO:tasks.rados.rados.0.smithi017.stdout:972: expect (ObjNum 457 snap 103 seq_num 457) 2023-03-29T14:06:28.222 INFO:tasks.rados.rados.0.smithi017.stdout:973: snap_remove snap 110 2023-03-29T14:06:29.225 INFO:tasks.rados.rados.0.smithi017.stdout:973: done (1 left) 2023-03-29T14:06:29.225 INFO:tasks.rados.rados.0.smithi017.stdout:974: delete oid 28 current snap is 111 2023-03-29T14:06:29.225 INFO:tasks.rados.rados.0.smithi017.stdout:972: expect (ObjNum 457 snap 103 seq_num 457) 2023-03-29T14:06:29.293 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:29.425 INFO:tasks.rados.rados.0.smithi017.stdout:972: done (1 left) 2023-03-29T14:06:29.425 INFO:tasks.rados.rados.0.smithi017.stdout:974: done (0 left) 2023-03-29T14:06:29.426 INFO:tasks.rados.rados.0.smithi017.stdout:975: snap_remove snap 109 2023-03-29T14:06:29.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:29.432+0000 7f9b26bb7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:29.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:29.436+0000 7f9b26bb7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:29.498 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:29.501 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:29.502 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:30.160 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:30.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:06:30.260 INFO:tasks.rados.rados.0.smithi017.stdout:975: done (0 left) 2023-03-29T14:06:30.260 INFO:tasks.rados.rados.0.smithi017.stdout:976: delete oid 19 current snap is 111 2023-03-29T14:06:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:30.289+0000 7f78ba6df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:30.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:30.298+0000 7f78ba6df700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:30.358 INFO:tasks.rados.rados.0.smithi017.stdout:976: done (0 left) 2023-03-29T14:06:30.358 INFO:tasks.rados.rados.0.smithi017.stdout:977: snap_remove snap 108 2023-03-29T14:06:31.263 INFO:tasks.rados.rados.0.smithi017.stdout:977: done (0 left) 2023-03-29T14:06:31.263 INFO:tasks.rados.rados.0.smithi017.stdout:978: snap_remove snap 107 2023-03-29T14:06:31.266 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:06:31.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:06:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:31.407+0000 7f2a1ba54700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:31.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:31.415+0000 7f2a1ba54700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:31.503 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:31.632+0000 7f68dff2a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:31.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:31.637+0000 7f68dff2a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:31.702 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:31.702 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:32.267 INFO:tasks.rados.rados.0.smithi017.stdout:978: done (0 left) 2023-03-29T14:06:32.268 INFO:tasks.rados.rados.0.smithi017.stdout:979: delete oid 46 current snap is 111 2023-03-29T14:06:32.271 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:06:32.284 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:06:32.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:32.413+0000 7f4af9e4d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:32.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:32.423+0000 7f4af9e4d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:32.460 INFO:tasks.rados.rados.0.smithi017.stdout:979: done (0 left) 2023-03-29T14:06:32.461 INFO:tasks.rados.rados.0.smithi017.stdout:980: write (excl) oid 37 current snap is 111 2023-03-29T14:06:32.461 INFO:tasks.rados.rados.0.smithi017.stdout:980: seq_num 483 ranges {517781=475506,1731038=386743} 2023-03-29T14:06:32.466 INFO:tasks.rados.rados.0.smithi017.stdout:980: writing smithi017107884-37 from 517781 to 993287 tid 1 2023-03-29T14:06:32.472 INFO:tasks.rados.rados.0.smithi017.stdout:980: writing smithi017107884-37 from 1731038 to 2117781 tid 2 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:981: read oid 29 snap -1 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:981: expect (ObjNum 429 snap 98 seq_num 429) 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:982: read oid 10 snap -1 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:982: expect (ObjNum 477 snap 109 seq_num 477) 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:983: read oid 14 snap -1 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:983: expect (ObjNum 454 snap 102 seq_num 454) 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:984: write (excl) oid 34 current snap is 111 2023-03-29T14:06:32.473 INFO:tasks.rados.rados.0.smithi017.stdout:984: seq_num 484 ranges {534588=550755,1789520=345068} 2023-03-29T14:06:32.478 INFO:tasks.rados.rados.0.smithi017.stdout:984: writing smithi017107884-34 from 534588 to 1085343 tid 1 2023-03-29T14:06:32.484 INFO:tasks.rados.rados.0.smithi017.stdout:984: writing smithi017107884-34 from 1789520 to 2134588 tid 2 2023-03-29T14:06:32.484 INFO:tasks.rados.rados.0.smithi017.stdout:985: read oid 47 snap -1 2023-03-29T14:06:32.485 INFO:tasks.rados.rados.0.smithi017.stdout:985: expect (ObjNum 469 snap 107 seq_num 469) 2023-03-29T14:06:32.485 INFO:tasks.rados.rados.0.smithi017.stdout:986: write (excl) oid 20 current snap is 111 2023-03-29T14:06:32.485 INFO:tasks.rados.rados.0.smithi017.stdout:986: seq_num 485 ranges {551395=626004,1848002=303393} 2023-03-29T14:06:32.491 INFO:tasks.rados.rados.0.smithi017.stdout:986: writing smithi017107884-20 from 551395 to 1177399 tid 1 2023-03-29T14:06:32.497 INFO:tasks.rados.rados.0.smithi017.stdout:986: writing smithi017107884-20 from 1848002 to 2151395 tid 2 2023-03-29T14:06:32.497 INFO:tasks.rados.rados.0.smithi017.stdout:987: read oid 8 snap -1 2023-03-29T14:06:32.497 INFO:tasks.rados.rados.0.smithi017.stdout:987: expect (ObjNum 463 snap 104 seq_num 463) 2023-03-29T14:06:32.497 INFO:tasks.rados.rados.0.smithi017.stdout:988: write oid 45 current snap is 111 2023-03-29T14:06:32.497 INFO:tasks.rados.rados.0.smithi017.stdout:988: seq_num 486 ranges {568202=701253,1906484=261718} 2023-03-29T14:06:32.504 INFO:tasks.rados.rados.0.smithi017.stdout:988: writing smithi017107884-45 from 568202 to 1269455 tid 1 2023-03-29T14:06:32.510 INFO:tasks.rados.rados.0.smithi017.stdout:988: writing smithi017107884-45 from 1906484 to 2168202 tid 2 2023-03-29T14:06:32.510 INFO:tasks.rados.rados.0.smithi017.stdout:989: delete oid 21 current snap is 111 2023-03-29T14:06:32.526 INFO:tasks.rados.rados.0.smithi017.stdout:980: finishing write tid 1 to smithi017107884-37 2023-03-29T14:06:32.537 INFO:tasks.rados.rados.0.smithi017.stdout:982: expect (ObjNum 477 snap 109 seq_num 477) 2023-03-29T14:06:32.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~83s 2023-03-29T14:06:32.697 INFO:tasks.rados.rados.0.smithi017.stdout:985: expect (ObjNum 469 snap 107 seq_num 469) 2023-03-29T14:06:32.845 INFO:tasks.rados.rados.0.smithi017.stdout:983: expect (ObjNum 454 snap 102 seq_num 454) 2023-03-29T14:06:33.022 INFO:tasks.rados.rados.0.smithi017.stdout:981: expect (ObjNum 429 snap 98 seq_num 429) 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:980: finishing write tid 2 to smithi017107884-37 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:984: finishing write tid 1 to smithi017107884-34 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:981: done (9 left) 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:982: done (8 left) 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:983: done (7 left) 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:985: done (6 left) 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:989: done (5 left) 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:990: write oid 14 current snap is 111 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:984: finishing write tid 2 to smithi017107884-34 2023-03-29T14:06:33.182 INFO:tasks.rados.rados.0.smithi017.stdout:990: seq_num 488 ranges {601816=568104,1623448=451823} 2023-03-29T14:06:33.190 INFO:tasks.rados.rados.0.smithi017.stdout:990: writing smithi017107884-14 from 601816 to 1169920 tid 1 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:990: writing smithi017107884-14 from 1623448 to 2075271 tid 2 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:991: delete oid 3 current snap is 111 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:980: finishing write tid 3 to smithi017107884-37 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:980: finishing write tid 4 to smithi017107884-37 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:980: oid 37 updating version 0 to 45 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:980: oid 37 version 45 is already newer than 43 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:980: oid 37 version 45 is already newer than 45 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 45 (ObjNum 483 snap 111 seq_num 483) dirty exists 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:980: left oid 37 (ObjNum 483 snap 111 seq_num 483) 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:984: finishing write tid 3 to smithi017107884-34 2023-03-29T14:06:33.197 INFO:tasks.rados.rados.0.smithi017.stdout:987: expect (ObjNum 463 snap 104 seq_num 463) 2023-03-29T14:06:33.272 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:06:33.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:06:33.399 INFO:tasks.rados.rados.0.smithi017.stdout:984: finishing write tid 4 to smithi017107884-34 2023-03-29T14:06:33.399 INFO:tasks.rados.rados.0.smithi017.stdout:984: oid 34 updating version 0 to 139 2023-03-29T14:06:33.399 INFO:tasks.rados.rados.0.smithi017.stdout:984: oid 34 version 139 is already newer than 139 2023-03-29T14:06:33.399 INFO:tasks.rados.rados.0.smithi017.stdout:984: oid 34 version 139 is already newer than 137 2023-03-29T14:06:33.399 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 139 (ObjNum 484 snap 111 seq_num 484) dirty exists 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:984: left oid 34 (ObjNum 484 snap 111 seq_num 484) 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: finishing write tid 1 to smithi017107884-20 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:980: done (6 left) 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:984: done (5 left) 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:987: done (4 left) 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:991: done (3 left) 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:992: snap_create 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: finishing write tid 2 to smithi017107884-20 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: finishing write tid 3 to smithi017107884-20 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: finishing write tid 4 to smithi017107884-20 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: oid 20 updating version 0 to 142 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: oid 20 version 142 is already newer than 140 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: oid 20 version 142 is already newer than 142 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 142 (ObjNum 485 snap 111 seq_num 485) dirty exists 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:986: left oid 20 (ObjNum 485 snap 111 seq_num 485) 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:988: finishing write tid 1 to smithi017107884-45 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:988: finishing write tid 2 to smithi017107884-45 2023-03-29T14:06:33.400 INFO:tasks.rados.rados.0.smithi017.stdout:988: finishing write tid 3 to smithi017107884-45 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:988: finishing write tid 4 to smithi017107884-45 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:988: oid 45 updating version 0 to 145 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:988: oid 45 version 145 is already newer than 145 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:988: oid 45 version 145 is already newer than 145 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 145 (ObjNum 486 snap 111 seq_num 486) dirty exists 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:988: left oid 45 (ObjNum 486 snap 111 seq_num 486) 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: finishing write tid 1 to smithi017107884-14 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: finishing write tid 2 to smithi017107884-14 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: finishing write tid 3 to smithi017107884-14 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: finishing write tid 4 to smithi017107884-14 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: oid 14 updating version 0 to 94 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: oid 14 version 94 is already newer than 94 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: oid 14 version 94 is already newer than 94 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 94 (ObjNum 488 snap 111 seq_num 488) dirty exists 2023-03-29T14:06:33.401 INFO:tasks.rados.rados.0.smithi017.stdout:990: left oid 14 (ObjNum 488 snap 111 seq_num 488) 2023-03-29T14:06:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:33.411+0000 7f0a10b44700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:33.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:33.420+0000 7f0a10b44700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:33.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:33.832 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:33.830+0000 7f3ab731c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:33.835 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:33.835+0000 7f3ab731c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:33.897 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:33.900 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:33.900 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:34.275 INFO:tasks.rados.rados.0.smithi017.stdout:986: done (3 left) 2023-03-29T14:06:34.275 INFO:tasks.rados.rados.0.smithi017.stdout:988: done (2 left) 2023-03-29T14:06:34.275 INFO:tasks.rados.rados.0.smithi017.stdout:990: done (1 left) 2023-03-29T14:06:34.275 INFO:tasks.rados.rados.0.smithi017.stdout:992: done (0 left) 2023-03-29T14:06:34.275 INFO:tasks.rados.rados.0.smithi017.stdout:993: write (excl) oid 41 current snap is 112 2023-03-29T14:06:34.275 INFO:tasks.rados.rados.0.smithi017.stdout:993: seq_num 490 ranges {635430=718602,2140412=95018} 2023-03-29T14:06:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:06:34.284 INFO:tasks.rados.rados.0.smithi017.stdout:993: writing smithi017107884-41 from 635430 to 1354032 tid 1 2023-03-29T14:06:34.289 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:06:34.289 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:993: writing smithi017107884-41 from 2140412 to 2235430 tid 2 2023-03-29T14:06:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:994: read oid 17 snap -1 2023-03-29T14:06:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:994: expect deleted 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:995: read oid 39 snap -1 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:995: expect deleted 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:996: read oid 18 snap -1 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:996: expect deleted 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:997: read oid 6 snap 111 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:997: expect (ObjNum 441 snap 99 seq_num 441) 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:998: read oid 30 snap 111 2023-03-29T14:06:34.293 INFO:tasks.rados.rados.0.smithi017.stdout:998: expect (ObjNum 370 snap 90 seq_num 370) 2023-03-29T14:06:34.294 INFO:tasks.rados.rados.0.smithi017.stdout:999: read oid 1 snap -1 2023-03-29T14:06:34.294 INFO:tasks.rados.rados.0.smithi017.stdout:999: expect deleted 2023-03-29T14:06:34.294 INFO:tasks.rados.rados.0.smithi017.stdout:1000: read oid 23 snap -1 2023-03-29T14:06:34.294 INFO:tasks.rados.rados.0.smithi017.stdout:1000: expect (ObjNum 440 snap 99 seq_num 440) 2023-03-29T14:06:34.294 INFO:tasks.rados.rados.0.smithi017.stdout:1001: write (excl) oid 34 current snap is 112 2023-03-29T14:06:34.294 INFO:tasks.rados.rados.0.smithi017.stdout:1001: seq_num 491 ranges {652237=793851,2198894=53343} 2023-03-29T14:06:34.301 INFO:tasks.rados.rados.0.smithi017.stdout:1001: writing smithi017107884-34 from 652237 to 1446088 tid 1 2023-03-29T14:06:34.306 INFO:tasks.rados.rados.0.smithi017.stdout:1001: writing smithi017107884-34 from 2198894 to 2252237 tid 2 2023-03-29T14:06:34.306 INFO:tasks.rados.rados.0.smithi017.stdout:1002: write oid 45 current snap is 112 2023-03-29T14:06:34.307 INFO:tasks.rados.rados.0.smithi017.stdout:1002: seq_num 492 ranges {669044=469100,1741023=459161} 2023-03-29T14:06:34.315 INFO:tasks.rados.rados.0.smithi017.stdout:1002: writing smithi017107884-45 from 669044 to 1138144 tid 1 2023-03-29T14:06:34.322 INFO:tasks.rados.rados.0.smithi017.stdout:1002: writing smithi017107884-45 from 1741023 to 2200184 tid 2 2023-03-29T14:06:34.323 INFO:tasks.rados.rados.0.smithi017.stdout:1003: read oid 4 snap -1 2023-03-29T14:06:34.323 INFO:tasks.rados.rados.0.smithi017.stdout:1003: expect (ObjNum 419 snap 97 seq_num 419) 2023-03-29T14:06:34.323 INFO:tasks.rados.rados.0.smithi017.stdout:1004: read oid 44 snap -1 2023-03-29T14:06:34.323 INFO:tasks.rados.rados.0.smithi017.stdout:1004: expect (ObjNum 472 snap 109 seq_num 472) 2023-03-29T14:06:34.323 INFO:tasks.rados.rados.0.smithi017.stdout:1005: delete oid 36 current snap is 112 2023-03-29T14:06:34.326 INFO:tasks.rados.rados.0.smithi017.stdout:997: expect (ObjNum 441 snap 99 seq_num 441) 2023-03-29T14:06:34.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:34.435+0000 7ff7e9c5b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:34.442 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:34.443+0000 7ff7e9c5b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:34.504 INFO:tasks.rados.rados.0.smithi017.stdout:1003: expect (ObjNum 419 snap 97 seq_num 419) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:994: done (12 left) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:995: done (11 left) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:996: done (10 left) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:997: done (9 left) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:999: done (8 left) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:1003: done (7 left) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:1005: done (6 left) 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:1006: write (excl) oid 4 current snap is 112 2023-03-29T14:06:34.627 INFO:tasks.rados.rados.0.smithi017.stdout:998: expect (ObjNum 370 snap 90 seq_num 370) 2023-03-29T14:06:34.672 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:34.674 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:34.687 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2023-03-29T14:06:34.273376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":188280484,"num_objects":548,"num_object_clones":452,"num_object_copies":1644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2305,"num_read_kb":1274026,"num_write":3120,"num_write_kb":837420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1643,"ondisk_log_size":1643,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1125888,"kb_used_data":1068576,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":376361472,"statfs":{"total":386547056640,"available":385394147328,"internally_reserved":0,"allocated":1094221824,"data_stored":1311310388,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6050322,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":10791,"num_write":44,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015073"},"pg_stats":[{"pgid":"3.f","version":"222'66","reported_seq":128,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.268941+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:30.268941+0000","last_peered":"2023-03-29T14:06:30.268941+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:30.268941+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345022,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":21195,"num_write":52,"num_write_kb":14796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"240'29","reported_seq":257,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.338425+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:29.338425+0000","last_peered":"2023-03-29T14:06:29.338425+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:29.338425+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":25429,"num_write":54,"num_write_kb":16442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"240'139","reported_seq":506,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.282118+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:30.282118+0000","last_peered":"2023-03-29T14:06:30.282118+0000","last_clean":"2023-03-29T14:06:30.282118+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:30.282118+0000","last_undegraded":"2023-03-29T14:06:30.282118+0000","last_fullsized":"2023-03-29T14:06:30.282118+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17815882,"num_objects":46,"num_object_clones":36,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":191,"num_read_kb":128302,"num_write":264,"num_write_kb":77648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"221'144","reported_seq":526,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.271286+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:30.271286+0000","last_peered":"2023-03-29T14:06:30.271286+0000","last_clean":"2023-03-29T14:06:30.271286+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:30.271286+0000","last_undegraded":"2023-03-29T14:06:30.271286+0000","last_fullsized":"2023-03-29T14:06:30.271286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521710,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":222,"num_read_kb":134489,"num_write":284,"num_write_kb":79564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"241'135","reported_seq":492,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.341306+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:29.341306+0000","last_peered":"2023-03-29T14:06:29.341306+0000","last_clean":"2023-03-29T14:06:29.341306+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:29.341306+0000","last_undegraded":"2023-03-29T14:06:29.341306+0000","last_fullsized":"2023-03-29T14:06:29.341306+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11328248,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":186,"num_read_kb":110007,"num_write":270,"num_write_kb":80482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":202,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.280543+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:30.280543+0000","last_peered":"2023-03-29T14:06:30.280543+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:30.280543+0000","last_undegraded":"2023-03-29T14:06:30.280543+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"228'91","reported_seq":395,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.275311+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:30.275311+0000","last_peered":"2023-03-29T14:06:30.275311+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:30.275311+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":112,"num_read_kb":51658,"num_write":164,"num_write_kb":48718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"243'66","reported_seq":342,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.270710+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:30.270710+0000","last_peered":"2023-03-29T14:06:30.270710+0000","last_clean":"2023-03-29T14:06:30.270710+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:30.270710+0000","last_undegraded":"2023-03-29T14:06:30.270710+0000","last_fullsized":"2023-03-29T14:06:30.270710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":87,"num_read_kb":49142,"num_write":126,"num_write_kb":35664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.313334+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:29.313334+0000","last_peered":"2023-03-29T14:06:29.313334+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:29.313334+0000","last_undegraded":"2023-03-29T14:06:29.313334+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":67,"reported_epoch":244,"state":"active+undersized","last_fresh":"2023-03-29T14:06:28.293669+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:28.293669+0000","last_peered":"2023-03-29T14:06:28.293669+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:28.293669+0000","last_undegraded":"2023-03-29T14:06:28.293669+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7262,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.278615+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:30.278615+0000","last_peered":"2023-03-29T14:06:30.278615+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:30.278615+0000","last_undegraded":"2023-03-29T14:06:30.278615+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"222'78","reported_seq":369,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:28.286612+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:28.286612+0000","last_peered":"2023-03-29T14:06:28.286612+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:28.286612+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13109724,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":70994,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.349373+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:29.349373+0000","last_peered":"2023-03-29T14:06:29.349373+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:29.349373+0000","last_undegraded":"2023-03-29T14:06:29.349373+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"237'91","reported_seq":390,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.298265+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:29.298265+0000","last_peered":"2023-03-29T14:06:29.298265+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:29.298265+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":110,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.267082+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:30.267082+0000","last_peered":"2023-03-29T14:06:30.267082+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:30.267082+0000","last_undegraded":"2023-03-29T14:06:30.267082+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":421,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.321686+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:29.321686+0000","last_peered":"2023-03-29T14:06:29.321686+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:29.321686+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"240'102","reported_seq":470,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.276541+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:30.276541+0000","last_peered":"2023-03-29T14:06:30.276541+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:30.276541+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25648076,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":218,"num_read_kb":142377,"num_write":206,"num_write_kb":59936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.273486+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:30.273486+0000","last_peered":"2023-03-29T14:06:30.273486+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:30.273486+0000","last_undegraded":"2023-03-29T14:06:30.273486+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"240'201","reported_seq":664,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:28.261279+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:28.261279+0000","last_peered":"2023-03-29T14:06:28.261279+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:28.261279+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21984150,"num_objects":70,"num_object_clones":56,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":347,"num_read_kb":214576,"num_write":372,"num_write_kb":107760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.369334+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:29.369334+0000","last_peered":"2023-03-29T14:06:29.369334+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:29.369334+0000","last_undegraded":"2023-03-29T14:06:29.369334+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"240'74","reported_seq":361,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.327320+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:29.327320+0000","last_peered":"2023-03-29T14:06:29.327320+0000","last_clean":"2023-03-29T14:06:29.327320+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:29.327320+0000","last_undegraded":"2023-03-29T14:06:29.327320+0000","last_fullsized":"2023-03-29T14:06:29.327320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12336602,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":59596,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"240'147","reported_seq":507,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.330485+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:29.330485+0000","last_peered":"2023-03-29T14:06:29.330485+0000","last_clean":"2023-03-29T14:06:29.330485+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:29.330485+0000","last_undegraded":"2023-03-29T14:06:29.330485+0000","last_fullsized":"2023-03-29T14:06:29.330485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26421132,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":177,"num_read_kb":98005,"num_write":282,"num_write_kb":84122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.335419+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:29.335419+0000","last_peered":"2023-03-29T14:06:29.335419+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:29.335419+0000","last_undegraded":"2023-03-29T14:06:29.335419+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.277871+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:30.277871+0000","last_peered":"2023-03-29T14:06:30.277871+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:30.277871+0000","last_undegraded":"2023-03-29T14:06:30.277871+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"237'141","reported_seq":506,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.324253+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:30.324253+0000","last_peered":"2023-03-29T14:06:30.324253+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:30.324253+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14891266,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":190,"num_read_kb":110001,"num_write":274,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":187099748,"num_objects":544,"num_object_clones":452,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2163,"num_read_kb":1273906,"num_write":2846,"num_write_kb":833632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1546,"ondisk_log_size":1546,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302208,"kb_used_data":287800,"kb_used_omap":0,"kb_used_meta":14400,"kb_avail":94069632,"statfs":{"total":96636764160,"available":96327303168,"internally_reserved":0,"allocated":294707200,"data_stored":349461303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14745600},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":343960,"kb_used_data":328400,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94027880,"statfs":{"total":96636764160,"available":96284549120,"internally_reserved":0,"allocated":336281600,"data_stored":406394364,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":323776,"kb_used_data":308536,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94048064,"statfs":{"total":96636764160,"available":96305217536,"internally_reserved":0,"allocated":315940864,"data_stored":382222496,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:34.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:34.714 INFO:tasks.rados.rados.0.smithi017.stdout:993: finishing write tid 1 to smithi017107884-41 2023-03-29T14:06:34.714 INFO:tasks.rados.rados.0.smithi017.stdout:1000: expect (ObjNum 440 snap 99 seq_num 440) 2023-03-29T14:06:34.847 INFO:tasks.rados.rados.0.smithi017.stdout:1001: finishing write tid 1 to smithi017107884-34 2023-03-29T14:06:34.847 INFO:tasks.rados.rados.0.smithi017.stdout:1004: expect (ObjNum 472 snap 109 seq_num 472) 2023-03-29T14:06:34.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:34.849+0000 7f0faa927700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:34.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:34.856+0000 7f0faa927700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:35.104 INFO:tasks.rados.rados.0.smithi017.stdout:993: finishing write tid 2 to smithi017107884-41 2023-03-29T14:06:35.104 INFO:tasks.rados.rados.0.smithi017.stdout:993: finishing write tid 3 to smithi017107884-41 2023-03-29T14:06:35.104 INFO:tasks.rados.rados.0.smithi017.stdout:993: finishing write tid 4 to smithi017107884-41 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:993: oid 41 updating version 0 to 150 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:993: oid 41 version 150 is already newer than 150 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:993: oid 41 version 150 is already newer than 148 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 150 (ObjNum 490 snap 112 seq_num 490) dirty exists 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:993: left oid 41 (ObjNum 490 snap 112 seq_num 490) 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1001: finishing write tid 2 to smithi017107884-34 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1001: finishing write tid 3 to smithi017107884-34 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1001: finishing write tid 4 to smithi017107884-34 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1001: oid 34 updating version 0 to 146 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1001: oid 34 updating version 146 to 149 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1001: oid 34 version 149 is already newer than 149 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 149 (ObjNum 491 snap 112 seq_num 491) dirty exists 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1001: left oid 34 (ObjNum 491 snap 112 seq_num 491) 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1002: finishing write tid 1 to smithi017107884-45 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1002: finishing write tid 2 to smithi017107884-45 2023-03-29T14:06:35.105 INFO:tasks.rados.rados.0.smithi017.stdout:1006: seq_num 494 ranges {702658=619598,1857987=444671} 2023-03-29T14:06:35.113 INFO:tasks.rados.rados.0.smithi017.stdout:1006: writing smithi017107884-4 from 702658 to 1322256 tid 1 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:1006: writing smithi017107884-4 from 1857987 to 2302658 tid 2 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:993: done (6 left) 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:998: done (5 left) 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:1000: done (4 left) 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:1001: done (3 left) 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:1004: done (2 left) 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:1007: read oid 49 snap 111 2023-03-29T14:06:35.117 INFO:tasks.rados.rados.0.smithi017.stdout:1007: expect deleted 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:1008: delete oid 10 current snap is 112 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:1002: finishing write tid 3 to smithi017107884-45 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:1002: finishing write tid 4 to smithi017107884-45 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:1002: oid 45 updating version 0 to 153 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:1002: oid 45 version 153 is already newer than 153 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:1002: oid 45 version 153 is already newer than 153 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 153 (ObjNum 492 snap 112 seq_num 492) dirty exists 2023-03-29T14:06:35.118 INFO:tasks.rados.rados.0.smithi017.stdout:1002: left oid 45 (ObjNum 492 snap 112 seq_num 492) 2023-03-29T14:06:35.162 INFO:tasks.rados.rados.0.smithi017.stdout:1006: finishing write tid 1 to smithi017107884-4 2023-03-29T14:06:35.213 INFO:tasks.rados.rados.0.smithi017.stdout:1006: finishing write tid 2 to smithi017107884-4 2023-03-29T14:06:35.214 INFO:tasks.rados.rados.0.smithi017.stdout:1006: finishing write tid 3 to smithi017107884-4 2023-03-29T14:06:35.216 INFO:tasks.rados.rados.0.smithi017.stdout:1006: finishing write tid 4 to smithi017107884-4 2023-03-29T14:06:35.216 INFO:tasks.rados.rados.0.smithi017.stdout:1006: oid 4 updating version 0 to 143 2023-03-29T14:06:35.216 INFO:tasks.rados.rados.0.smithi017.stdout:1006: oid 4 version 143 is already newer than 140 2023-03-29T14:06:35.216 INFO:tasks.rados.rados.0.smithi017.stdout:1006: oid 4 version 143 is already newer than 143 2023-03-29T14:06:35.216 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 143 (ObjNum 494 snap 112 seq_num 494) dirty exists 2023-03-29T14:06:35.216 INFO:tasks.rados.rados.0.smithi017.stdout:1006: left oid 4 (ObjNum 494 snap 112 seq_num 494) 2023-03-29T14:06:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:1002: done (3 left) 2023-03-29T14:06:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:1006: done (2 left) 2023-03-29T14:06:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:1007: done (1 left) 2023-03-29T14:06:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:1008: done (0 left) 2023-03-29T14:06:35.262 INFO:tasks.rados.rados.0.smithi017.stdout:1009: snap_remove snap 111 2023-03-29T14:06:35.274 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:35.276 INFO:tasks.rados.rados.0.smithi017.stdout:1009: done (0 left) 2023-03-29T14:06:35.276 INFO:tasks.rados.rados.0.smithi017.stdout:1010: write oid 49 current snap is 112 2023-03-29T14:06:35.276 INFO:tasks.rados.rados.0.smithi017.stdout:1010: seq_num 496 ranges {736272=486449,1974951=361321} 2023-03-29T14:06:35.276 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:35.282 INFO:tasks.rados.rados.0.smithi017.stdout:1010: writing smithi017107884-49 from 736272 to 1222721 tid 1 2023-03-29T14:06:35.288 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2023-03-29T14:06:34.273376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":188280484,"num_objects":548,"num_object_clones":452,"num_object_copies":1644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2305,"num_read_kb":1274026,"num_write":3120,"num_write_kb":837420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1643,"ondisk_log_size":1643,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1125888,"kb_used_data":1068576,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":376361472,"statfs":{"total":386547056640,"available":385394147328,"internally_reserved":0,"allocated":1094221824,"data_stored":1311310388,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6050322,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":10791,"num_write":44,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015073"},"pg_stats":[{"pgid":"3.f","version":"222'66","reported_seq":128,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.268941+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:30.268941+0000","last_peered":"2023-03-29T14:06:30.268941+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:30.268941+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345022,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":21195,"num_write":52,"num_write_kb":14796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"240'29","reported_seq":257,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.338425+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:29.338425+0000","last_peered":"2023-03-29T14:06:29.338425+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:29.338425+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":25429,"num_write":54,"num_write_kb":16442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"240'139","reported_seq":506,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.282118+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:30.282118+0000","last_peered":"2023-03-29T14:06:30.282118+0000","last_clean":"2023-03-29T14:06:30.282118+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:30.282118+0000","last_undegraded":"2023-03-29T14:06:30.282118+0000","last_fullsized":"2023-03-29T14:06:30.282118+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17815882,"num_objects":46,"num_object_clones":36,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":191,"num_read_kb":128302,"num_write":264,"num_write_kb":77648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"221'144","reported_seq":526,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.271286+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:30.271286+0000","last_peered":"2023-03-29T14:06:30.271286+0000","last_clean":"2023-03-29T14:06:30.271286+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:30.271286+0000","last_undegraded":"2023-03-29T14:06:30.271286+0000","last_fullsized":"2023-03-29T14:06:30.271286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521710,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":222,"num_read_kb":134489,"num_write":284,"num_write_kb":79564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"241'135","reported_seq":492,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.341306+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:29.341306+0000","last_peered":"2023-03-29T14:06:29.341306+0000","last_clean":"2023-03-29T14:06:29.341306+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:29.341306+0000","last_undegraded":"2023-03-29T14:06:29.341306+0000","last_fullsized":"2023-03-29T14:06:29.341306+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11328248,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":186,"num_read_kb":110007,"num_write":270,"num_write_kb":80482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":202,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.280543+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:30.280543+0000","last_peered":"2023-03-29T14:06:30.280543+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:30.280543+0000","last_undegraded":"2023-03-29T14:06:30.280543+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"228'91","reported_seq":395,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.275311+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:30.275311+0000","last_peered":"2023-03-29T14:06:30.275311+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:30.275311+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":112,"num_read_kb":51658,"num_write":164,"num_write_kb":48718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"243'66","reported_seq":342,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.270710+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:30.270710+0000","last_peered":"2023-03-29T14:06:30.270710+0000","last_clean":"2023-03-29T14:06:30.270710+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:30.270710+0000","last_undegraded":"2023-03-29T14:06:30.270710+0000","last_fullsized":"2023-03-29T14:06:30.270710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":87,"num_read_kb":49142,"num_write":126,"num_write_kb":35664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.313334+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:29.313334+0000","last_peered":"2023-03-29T14:06:29.313334+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:29.313334+0000","last_undegraded":"2023-03-29T14:06:29.313334+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":67,"reported_epoch":244,"state":"active+undersized","last_fresh":"2023-03-29T14:06:28.293669+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:28.293669+0000","last_peered":"2023-03-29T14:06:28.293669+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:28.293669+0000","last_undegraded":"2023-03-29T14:06:28.293669+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7262,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.278615+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:30.278615+0000","last_peered":"2023-03-29T14:06:30.278615+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:30.278615+0000","last_undegraded":"2023-03-29T14:06:30.278615+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"222'78","reported_seq":369,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:28.286612+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:28.286612+0000","last_peered":"2023-03-29T14:06:28.286612+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:28.286612+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13109724,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":70994,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.349373+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:29.349373+0000","last_peered":"2023-03-29T14:06:29.349373+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:29.349373+0000","last_undegraded":"2023-03-29T14:06:29.349373+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"237'91","reported_seq":390,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.298265+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:29.298265+0000","last_peered":"2023-03-29T14:06:29.298265+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:29.298265+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":110,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.267082+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:30.267082+0000","last_peered":"2023-03-29T14:06:30.267082+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:30.267082+0000","last_undegraded":"2023-03-29T14:06:30.267082+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":421,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.321686+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:29.321686+0000","last_peered":"2023-03-29T14:06:29.321686+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:29.321686+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"240'102","reported_seq":470,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.276541+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:30.276541+0000","last_peered":"2023-03-29T14:06:30.276541+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:30.276541+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25648076,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":218,"num_read_kb":142377,"num_write":206,"num_write_kb":59936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.273486+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:30.273486+0000","last_peered":"2023-03-29T14:06:30.273486+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:30.273486+0000","last_undegraded":"2023-03-29T14:06:30.273486+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"240'201","reported_seq":664,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:28.261279+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:28.261279+0000","last_peered":"2023-03-29T14:06:28.261279+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:28.261279+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21984150,"num_objects":70,"num_object_clones":56,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":347,"num_read_kb":214576,"num_write":372,"num_write_kb":107760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.369334+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:29.369334+0000","last_peered":"2023-03-29T14:06:29.369334+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:29.369334+0000","last_undegraded":"2023-03-29T14:06:29.369334+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"240'74","reported_seq":361,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.327320+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:29.327320+0000","last_peered":"2023-03-29T14:06:29.327320+0000","last_clean":"2023-03-29T14:06:29.327320+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:29.327320+0000","last_undegraded":"2023-03-29T14:06:29.327320+0000","last_fullsized":"2023-03-29T14:06:29.327320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12336602,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":59596,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"240'147","reported_seq":507,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.330485+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:29.330485+0000","last_peered":"2023-03-29T14:06:29.330485+0000","last_clean":"2023-03-29T14:06:29.330485+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:29.330485+0000","last_undegraded":"2023-03-29T14:06:29.330485+0000","last_fullsized":"2023-03-29T14:06:29.330485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26421132,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":177,"num_read_kb":98005,"num_write":282,"num_write_kb":84122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.335419+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:29.335419+0000","last_peered":"2023-03-29T14:06:29.335419+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:29.335419+0000","last_undegraded":"2023-03-29T14:06:29.335419+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.277871+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:30.277871+0000","last_peered":"2023-03-29T14:06:30.277871+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:30.277871+0000","last_undegraded":"2023-03-29T14:06:30.277871+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"237'141","reported_seq":506,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.324253+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:30.324253+0000","last_peered":"2023-03-29T14:06:30.324253+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:30.324253+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14891266,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":190,"num_read_kb":110001,"num_write":274,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":187099748,"num_objects":544,"num_object_clones":452,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2163,"num_read_kb":1273906,"num_write":2846,"num_write_kb":833632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1546,"ondisk_log_size":1546,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302208,"kb_used_data":287800,"kb_used_omap":0,"kb_used_meta":14400,"kb_avail":94069632,"statfs":{"total":96636764160,"available":96327303168,"internally_reserved":0,"allocated":294707200,"data_stored":349461303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14745600},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":343960,"kb_used_data":328400,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94027880,"statfs":{"total":96636764160,"available":96284549120,"internally_reserved":0,"allocated":336281600,"data_stored":406394364,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":323776,"kb_used_data":308536,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94048064,"statfs":{"total":96636764160,"available":96305217536,"internally_reserved":0,"allocated":315940864,"data_stored":382222496,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:35.291 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:06:35.292 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:35.292 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:35.292 INFO:tasks.rados.rados.0.smithi017.stdout:1010: writing smithi017107884-49 from 1974951 to 2336272 tid 2 2023-03-29T14:06:35.293 INFO:tasks.rados.rados.0.smithi017.stdout:1011: write (excl) oid 8 current snap is 112 2023-03-29T14:06:35.293 INFO:tasks.rados.rados.0.smithi017.stdout:1011: seq_num 497 ranges {753079=561698,2033433=319646} 2023-03-29T14:06:35.296 INFO:tasks.rados.rados.0.smithi017.stdout:1011: writing smithi017107884-8 from 753079 to 1314777 tid 1 2023-03-29T14:06:35.303 INFO:tasks.rados.rados.0.smithi017.stdout:1011: writing smithi017107884-8 from 2033433 to 2353079 tid 2 2023-03-29T14:06:35.303 INFO:tasks.rados.rados.0.smithi017.stdout:1012: read oid 18 snap -1 2023-03-29T14:06:35.303 INFO:tasks.rados.rados.0.smithi017.stdout:1012: expect deleted 2023-03-29T14:06:35.303 INFO:tasks.rados.rados.0.smithi017.stdout:1013: write oid 21 current snap is 112 2023-03-29T14:06:35.303 INFO:tasks.rados.rados.0.smithi017.stdout:1013: seq_num 498 ranges {769886=636947,2091915=277971} 2023-03-29T14:06:35.311 INFO:tasks.rados.rados.0.smithi017.stdout:1013: writing smithi017107884-21 from 769886 to 1406833 tid 1 2023-03-29T14:06:35.316 INFO:tasks.rados.rados.0.smithi017.stdout:1013: writing smithi017107884-21 from 2091915 to 2369886 tid 2 2023-03-29T14:06:35.317 INFO:tasks.rados.rados.0.smithi017.stdout:1014: write oid 5 current snap is 112 2023-03-29T14:06:35.317 INFO:tasks.rados.rados.0.smithi017.stdout:1014: seq_num 499 ranges {786693=712196,2150397=236296} 2023-03-29T14:06:35.325 INFO:tasks.rados.rados.0.smithi017.stdout:1014: writing smithi017107884-5 from 786693 to 1498889 tid 1 2023-03-29T14:06:35.330 INFO:tasks.rados.rados.0.smithi017.stdout:1014: writing smithi017107884-5 from 2150397 to 2386693 tid 2 2023-03-29T14:06:35.330 INFO:tasks.rados.rados.0.smithi017.stdout:1015: delete oid 6 current snap is 112 2023-03-29T14:06:35.332 INFO:tasks.rados.rados.0.smithi017.stdout:1013: finishing write tid 1 to smithi017107884-21 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:1013: finishing write tid 2 to smithi017107884-21 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:1013: finishing write tid 3 to smithi017107884-21 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:1013: finishing write tid 4 to smithi017107884-21 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:1013: oid 21 updating version 0 to 106 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:1013: oid 21 version 106 is already newer than 106 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:1013: oid 21 version 106 is already newer than 106 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 106 (ObjNum 498 snap 112 seq_num 498) dirty exists 2023-03-29T14:06:35.374 INFO:tasks.rados.rados.0.smithi017.stdout:1013: left oid 21 (ObjNum 498 snap 112 seq_num 498) 2023-03-29T14:06:35.407 INFO:tasks.rados.rados.0.smithi017.stdout:1010: finishing write tid 1 to smithi017107884-49 2023-03-29T14:06:35.412 INFO:tasks.rados.rados.0.smithi017.stdout:1010: finishing write tid 2 to smithi017107884-49 2023-03-29T14:06:35.414 INFO:tasks.rados.rados.0.smithi017.stdout:1010: finishing write tid 3 to smithi017107884-49 2023-03-29T14:06:35.417 INFO:tasks.rados.rados.0.smithi017.stdout:1010: finishing write tid 4 to smithi017107884-49 2023-03-29T14:06:35.417 INFO:tasks.rados.rados.0.smithi017.stdout:1010: oid 49 updating version 0 to 205 2023-03-29T14:06:35.417 INFO:tasks.rados.rados.0.smithi017.stdout:1010: oid 49 version 205 is already newer than 205 2023-03-29T14:06:35.417 INFO:tasks.rados.rados.0.smithi017.stdout:1010: oid 49 version 205 is already newer than 205 2023-03-29T14:06:35.418 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 205 (ObjNum 496 snap 112 seq_num 496) dirty exists 2023-03-29T14:06:35.418 INFO:tasks.rados.rados.0.smithi017.stdout:1010: left oid 49 (ObjNum 496 snap 112 seq_num 496) 2023-03-29T14:06:35.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:35.427+0000 7f8303612700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:35.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:35.434+0000 7f8303612700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:35.441 INFO:tasks.rados.rados.0.smithi017.stdout:1010: done (5 left) 2023-03-29T14:06:35.442 INFO:tasks.rados.rados.0.smithi017.stdout:1012: done (4 left) 2023-03-29T14:06:35.442 INFO:tasks.rados.rados.0.smithi017.stdout:1013: done (3 left) 2023-03-29T14:06:35.442 INFO:tasks.rados.rados.0.smithi017.stdout:1015: done (2 left) 2023-03-29T14:06:35.442 INFO:tasks.rados.rados.0.smithi017.stdout:1016: delete oid 42 current snap is 112 2023-03-29T14:06:35.494 INFO:tasks.rados.rados.0.smithi017.stdout:1014: finishing write tid 1 to smithi017107884-5 2023-03-29T14:06:35.496 INFO:tasks.rados.rados.0.smithi017.stdout:1011: finishing write tid 1 to smithi017107884-8 2023-03-29T14:06:35.496 INFO:tasks.rados.rados.0.smithi017.stdout:1016: done (2 left) 2023-03-29T14:06:35.496 INFO:tasks.rados.rados.0.smithi017.stdout:1017: delete oid 24 current snap is 112 2023-03-29T14:06:35.497 INFO:tasks.rados.rados.0.smithi017.stdout:1014: finishing write tid 2 to smithi017107884-5 2023-03-29T14:06:35.500 INFO:tasks.rados.rados.0.smithi017.stdout:1014: finishing write tid 3 to smithi017107884-5 2023-03-29T14:06:35.502 INFO:tasks.rados.rados.0.smithi017.stdout:1014: finishing write tid 4 to smithi017107884-5 2023-03-29T14:06:35.502 INFO:tasks.rados.rados.0.smithi017.stdout:1014: oid 5 updating version 0 to 153 2023-03-29T14:06:35.503 INFO:tasks.rados.rados.0.smithi017.stdout:1014: oid 5 version 153 is already newer than 153 2023-03-29T14:06:35.503 INFO:tasks.rados.rados.0.smithi017.stdout:1014: oid 5 version 153 is already newer than 153 2023-03-29T14:06:35.503 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 153 (ObjNum 499 snap 112 seq_num 499) dirty exists 2023-03-29T14:06:35.503 INFO:tasks.rados.rados.0.smithi017.stdout:1014: left oid 5 (ObjNum 499 snap 112 seq_num 499) 2023-03-29T14:06:35.531 INFO:tasks.rados.rados.0.smithi017.stdout:1011: finishing write tid 2 to smithi017107884-8 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1011: finishing write tid 3 to smithi017107884-8 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1011: finishing write tid 4 to smithi017107884-8 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1011: oid 8 updating version 0 to 145 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1011: oid 8 version 145 is already newer than 143 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1011: oid 8 version 145 is already newer than 145 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 145 (ObjNum 497 snap 112 seq_num 497) dirty exists 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1011: left oid 8 (ObjNum 497 snap 112 seq_num 497) 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1011: done (2 left) 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1014: done (1 left) 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1017: done (0 left) 2023-03-29T14:06:35.572 INFO:tasks.rados.rados.0.smithi017.stdout:1018: delete oid 47 current snap is 112 2023-03-29T14:06:35.621 INFO:tasks.rados.rados.0.smithi017.stdout:1018: done (0 left) 2023-03-29T14:06:35.621 INFO:tasks.rados.rados.0.smithi017.stdout:1019: delete oid 44 current snap is 112 2023-03-29T14:06:35.644 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:35.658 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2023-03-29T14:06:35.274252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":188280484,"num_objects":548,"num_object_clones":452,"num_object_copies":1644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2305,"num_read_kb":1274026,"num_write":3120,"num_write_kb":837420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1643,"ondisk_log_size":1643,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1125888,"kb_used_data":1068576,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":376361472,"statfs":{"total":386547056640,"available":385394147328,"internally_reserved":0,"allocated":1094221824,"data_stored":1311310388,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6050322,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":10791,"num_write":44,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012174"},"pg_stats":[{"pgid":"3.f","version":"222'66","reported_seq":128,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.268941+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:30.268941+0000","last_peered":"2023-03-29T14:06:30.268941+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:30.268941+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345022,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":21195,"num_write":52,"num_write_kb":14796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"240'29","reported_seq":257,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.338425+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:29.338425+0000","last_peered":"2023-03-29T14:06:29.338425+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:29.338425+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":25429,"num_write":54,"num_write_kb":16442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"240'139","reported_seq":506,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.282118+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:30.282118+0000","last_peered":"2023-03-29T14:06:30.282118+0000","last_clean":"2023-03-29T14:06:30.282118+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:30.282118+0000","last_undegraded":"2023-03-29T14:06:30.282118+0000","last_fullsized":"2023-03-29T14:06:30.282118+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17815882,"num_objects":46,"num_object_clones":36,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":191,"num_read_kb":128302,"num_write":264,"num_write_kb":77648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"221'144","reported_seq":526,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.271286+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:30.271286+0000","last_peered":"2023-03-29T14:06:30.271286+0000","last_clean":"2023-03-29T14:06:30.271286+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:30.271286+0000","last_undegraded":"2023-03-29T14:06:30.271286+0000","last_fullsized":"2023-03-29T14:06:30.271286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521710,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":222,"num_read_kb":134489,"num_write":284,"num_write_kb":79564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"241'135","reported_seq":492,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.341306+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:29.341306+0000","last_peered":"2023-03-29T14:06:29.341306+0000","last_clean":"2023-03-29T14:06:29.341306+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:29.341306+0000","last_undegraded":"2023-03-29T14:06:29.341306+0000","last_fullsized":"2023-03-29T14:06:29.341306+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11328248,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":186,"num_read_kb":110007,"num_write":270,"num_write_kb":80482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":202,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.280543+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:30.280543+0000","last_peered":"2023-03-29T14:06:30.280543+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:30.280543+0000","last_undegraded":"2023-03-29T14:06:30.280543+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"228'91","reported_seq":395,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.275311+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:30.275311+0000","last_peered":"2023-03-29T14:06:30.275311+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:30.275311+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260756,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":112,"num_read_kb":51658,"num_write":164,"num_write_kb":48718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"243'66","reported_seq":342,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-03-29T14:06:30.270710+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:30.270710+0000","last_peered":"2023-03-29T14:06:30.270710+0000","last_clean":"2023-03-29T14:06:30.270710+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:30.270710+0000","last_undegraded":"2023-03-29T14:06:30.270710+0000","last_fullsized":"2023-03-29T14:06:30.270710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":87,"num_read_kb":49142,"num_write":126,"num_write_kb":35664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.313334+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:29.313334+0000","last_peered":"2023-03-29T14:06:29.313334+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:29.313334+0000","last_undegraded":"2023-03-29T14:06:29.313334+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'42","reported_seq":67,"reported_epoch":244,"state":"active+undersized","last_fresh":"2023-03-29T14:06:28.293669+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:28.293669+0000","last_peered":"2023-03-29T14:06:28.293669+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:28.293669+0000","last_undegraded":"2023-03-29T14:06:28.293669+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4437180,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7262,"num_write":8,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.278615+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:30.278615+0000","last_peered":"2023-03-29T14:06:30.278615+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:30.278615+0000","last_undegraded":"2023-03-29T14:06:30.278615+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"222'78","reported_seq":369,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:28.286612+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:28.286612+0000","last_peered":"2023-03-29T14:06:28.286612+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:28.286612+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13109724,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":70994,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.349373+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:29.349373+0000","last_peered":"2023-03-29T14:06:29.349373+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:29.349373+0000","last_undegraded":"2023-03-29T14:06:29.349373+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"237'91","reported_seq":390,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.298265+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:29.298265+0000","last_peered":"2023-03-29T14:06:29.298265+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:29.298265+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":110,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.267082+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:30.267082+0000","last_peered":"2023-03-29T14:06:30.267082+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:30.267082+0000","last_undegraded":"2023-03-29T14:06:30.267082+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":421,"reported_epoch":243,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:29.321686+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:29.321686+0000","last_peered":"2023-03-29T14:06:29.321686+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:29.321686+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"240'102","reported_seq":470,"reported_epoch":245,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.276541+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:30.276541+0000","last_peered":"2023-03-29T14:06:30.276541+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:30.276541+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25648076,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":218,"num_read_kb":142377,"num_write":206,"num_write_kb":59936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.273486+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:30.273486+0000","last_peered":"2023-03-29T14:06:30.273486+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:30.273486+0000","last_undegraded":"2023-03-29T14:06:30.273486+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"240'201","reported_seq":664,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:28.261279+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:28.261279+0000","last_peered":"2023-03-29T14:06:28.261279+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:28.261279+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21984150,"num_objects":70,"num_object_clones":56,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":347,"num_read_kb":214576,"num_write":372,"num_write_kb":107760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.369334+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:29.369334+0000","last_peered":"2023-03-29T14:06:29.369334+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:29.369334+0000","last_undegraded":"2023-03-29T14:06:29.369334+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"240'74","reported_seq":361,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.327320+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:29.327320+0000","last_peered":"2023-03-29T14:06:29.327320+0000","last_clean":"2023-03-29T14:06:29.327320+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:29.327320+0000","last_undegraded":"2023-03-29T14:06:29.327320+0000","last_fullsized":"2023-03-29T14:06:29.327320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12336602,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":59596,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"240'147","reported_seq":507,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-03-29T14:06:29.330485+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:29.330485+0000","last_peered":"2023-03-29T14:06:29.330485+0000","last_clean":"2023-03-29T14:06:29.330485+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:29.330485+0000","last_undegraded":"2023-03-29T14:06:29.330485+0000","last_fullsized":"2023-03-29T14:06:29.330485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26421132,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":177,"num_read_kb":98005,"num_write":282,"num_write_kb":84122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":243,"state":"active+undersized","last_fresh":"2023-03-29T14:06:29.335419+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:29.335419+0000","last_peered":"2023-03-29T14:06:29.335419+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:29.335419+0000","last_undegraded":"2023-03-29T14:06:29.335419+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":245,"state":"active+undersized","last_fresh":"2023-03-29T14:06:30.277871+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:30.277871+0000","last_peered":"2023-03-29T14:06:30.277871+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:30.277871+0000","last_undegraded":"2023-03-29T14:06:30.277871+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"237'141","reported_seq":506,"reported_epoch":244,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:30.324253+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:30.324253+0000","last_peered":"2023-03-29T14:06:30.324253+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:30.324253+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14891266,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":190,"num_read_kb":110001,"num_write":274,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":187099748,"num_objects":544,"num_object_clones":452,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2163,"num_read_kb":1273906,"num_write":2846,"num_write_kb":833632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1546,"ondisk_log_size":1546,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302208,"kb_used_data":287800,"kb_used_omap":0,"kb_used_meta":14400,"kb_avail":94069632,"statfs":{"total":96636764160,"available":96327303168,"internally_reserved":0,"allocated":294707200,"data_stored":349461303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14745600},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":343960,"kb_used_data":328400,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94027880,"statfs":{"total":96636764160,"available":96284549120,"internally_reserved":0,"allocated":336281600,"data_stored":406394364,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":323776,"kb_used_data":308536,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94048064,"statfs":{"total":96636764160,"available":96305217536,"internally_reserved":0,"allocated":315940864,"data_stored":382222496,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:35.704 INFO:tasks.rados.rados.0.smithi017.stdout:1019: done (0 left) 2023-03-29T14:06:35.704 INFO:tasks.rados.rados.0.smithi017.stdout:1020: write (excl) oid 14 current snap is 112 2023-03-29T14:06:35.704 INFO:tasks.rados.rados.0.smithi017.stdout:1020: seq_num 505 ranges {487535=480043,1584936=624833} 2023-03-29T14:06:35.711 INFO:tasks.rados.rados.0.smithi017.stdout:1020: writing smithi017107884-14 from 487535 to 967578 tid 1 2023-03-29T14:06:35.720 INFO:tasks.rados.rados.0.smithi017.stdout:1020: writing smithi017107884-14 from 1584936 to 2209769 tid 2 2023-03-29T14:06:35.720 INFO:tasks.rados.rados.0.smithi017.stdout:1021: snap_create 2023-03-29T14:06:35.732 INFO:tasks.rados.rados.0.smithi017.stdout:1020: finishing write tid 1 to smithi017107884-14 2023-03-29T14:06:35.782 INFO:tasks.rados.rados.0.smithi017.stdout:1020: finishing write tid 2 to smithi017107884-14 2023-03-29T14:06:35.782 INFO:tasks.rados.rados.0.smithi017.stdout:1020: finishing write tid 3 to smithi017107884-14 2023-03-29T14:06:35.783 INFO:tasks.rados.rados.0.smithi017.stdout:1020: finishing write tid 4 to smithi017107884-14 2023-03-29T14:06:35.784 INFO:tasks.rados.rados.0.smithi017.stdout:1020: oid 14 updating version 0 to 95 2023-03-29T14:06:35.784 INFO:tasks.rados.rados.0.smithi017.stdout:1020: oid 14 updating version 95 to 97 2023-03-29T14:06:35.784 INFO:tasks.rados.rados.0.smithi017.stdout:1020: oid 14 version 97 is already newer than 97 2023-03-29T14:06:35.784 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 97 (ObjNum 505 snap 112 seq_num 505) dirty exists 2023-03-29T14:06:35.784 INFO:tasks.rados.rados.0.smithi017.stdout:1020: left oid 14 (ObjNum 505 snap 112 seq_num 505) 2023-03-29T14:06:35.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:36.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:36.029+0000 7f8b98eaf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:36.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:36.033+0000 7f8b98eaf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:36.095 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:36.098 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:36.098 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:36.285 INFO:tasks.rados.rados.0.smithi017.stdout:1020: done (1 left) 2023-03-29T14:06:36.285 INFO:tasks.rados.rados.0.smithi017.stdout:1021: done (0 left) 2023-03-29T14:06:36.285 INFO:tasks.rados.rados.0.smithi017.stdout:1022: delete oid 1 current snap is 113 2023-03-29T14:06:36.343 INFO:tasks.rados.rados.0.smithi017.stdout:1022: done (0 left) 2023-03-29T14:06:36.343 INFO:tasks.rados.rados.0.smithi017.stdout:1023: write oid 25 current snap is 113 2023-03-29T14:06:36.344 INFO:tasks.rados.rados.0.smithi017.stdout:1023: seq_num 507 ranges {521149=630541,1701900=628502} 2023-03-29T14:06:36.350 INFO:tasks.rados.rados.0.smithi017.stdout:1023: writing smithi017107884-25 from 521149 to 1151690 tid 1 2023-03-29T14:06:36.356 INFO:tasks.rados.rados.0.smithi017.stdout:1023: writing smithi017107884-25 from 1701900 to 2330402 tid 2 2023-03-29T14:06:36.356 INFO:tasks.rados.rados.0.smithi017.stdout:1024: write oid 37 current snap is 113 2023-03-29T14:06:36.356 INFO:tasks.rados.rados.0.smithi017.stdout:1024: seq_num 508 ranges {537956=705790,1644029=772160} 2023-03-29T14:06:36.363 INFO:tasks.rados.rados.0.smithi017.stdout:1024: writing smithi017107884-37 from 537956 to 1243746 tid 1 2023-03-29T14:06:36.370 INFO:tasks.rados.rados.0.smithi017.stdout:1024: writing smithi017107884-37 from 1644029 to 2416189 tid 2 2023-03-29T14:06:36.370 INFO:tasks.rados.rados.0.smithi017.stdout:1025: snap_create 2023-03-29T14:06:36.437 INFO:tasks.rados.rados.0.smithi017.stdout:1023: finishing write tid 1 to smithi017107884-25 2023-03-29T14:06:36.445 INFO:tasks.rados.rados.0.smithi017.stdout:1024: finishing write tid 1 to smithi017107884-37 2023-03-29T14:06:36.479 INFO:tasks.rados.rados.0.smithi017.stdout:1023: finishing write tid 2 to smithi017107884-25 2023-03-29T14:06:36.480 INFO:tasks.rados.rados.0.smithi017.stdout:1023: finishing write tid 3 to smithi017107884-25 2023-03-29T14:06:36.480 INFO:tasks.rados.rados.0.smithi017.stdout:1023: finishing write tid 4 to smithi017107884-25 2023-03-29T14:06:36.480 INFO:tasks.rados.rados.0.smithi017.stdout:1023: oid 25 updating version 0 to 149 2023-03-29T14:06:36.481 INFO:tasks.rados.rados.0.smithi017.stdout:1023: oid 25 version 149 is already newer than 149 2023-03-29T14:06:36.481 INFO:tasks.rados.rados.0.smithi017.stdout:1023: oid 25 version 149 is already newer than 149 2023-03-29T14:06:36.481 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 149 (ObjNum 507 snap 113 seq_num 507) dirty exists 2023-03-29T14:06:36.481 INFO:tasks.rados.rados.0.smithi017.stdout:1023: left oid 25 (ObjNum 507 snap 113 seq_num 507) 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:1024: finishing write tid 2 to smithi017107884-37 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:1024: finishing write tid 3 to smithi017107884-37 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:1024: finishing write tid 4 to smithi017107884-37 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:1024: oid 37 updating version 0 to 49 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:1024: oid 37 version 49 is already newer than 49 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:1024: oid 37 version 49 is already newer than 49 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 49 (ObjNum 508 snap 113 seq_num 508) dirty exists 2023-03-29T14:06:36.486 INFO:tasks.rados.rados.0.smithi017.stdout:1024: left oid 37 (ObjNum 508 snap 113 seq_num 508) 2023-03-29T14:06:37.290 INFO:tasks.rados.rados.0.smithi017.stdout:1023: done (2 left) 2023-03-29T14:06:37.290 INFO:tasks.rados.rados.0.smithi017.stdout:1024: done (1 left) 2023-03-29T14:06:37.290 INFO:tasks.rados.rados.0.smithi017.stdout:1025: done (0 left) 2023-03-29T14:06:37.290 INFO:tasks.rados.rados.0.smithi017.stdout:1026: write oid 23 current snap is 114 2023-03-29T14:06:37.290 INFO:tasks.rados.rados.0.smithi017.stdout:1026: seq_num 509 ranges {554763=781039,2102511=452252} 2023-03-29T14:06:37.297 INFO:tasks.rados.rados.0.smithi017.stdout:1026: writing smithi017107884-23 from 554763 to 1335802 tid 1 2023-03-29T14:06:37.303 INFO:tasks.rados.rados.0.smithi017.stdout:1026: writing smithi017107884-23 from 2102511 to 2554763 tid 2 2023-03-29T14:06:37.304 INFO:tasks.rados.rados.0.smithi017.stdout:1027: snap_create 2023-03-29T14:06:37.401 INFO:tasks.rados.rados.0.smithi017.stdout:1026: finishing write tid 1 to smithi017107884-23 2023-03-29T14:06:37.404 INFO:tasks.rados.rados.0.smithi017.stdout:1026: finishing write tid 2 to smithi017107884-23 2023-03-29T14:06:37.405 INFO:tasks.rados.rados.0.smithi017.stdout:1026: finishing write tid 3 to smithi017107884-23 2023-03-29T14:06:37.408 INFO:tasks.rados.rados.0.smithi017.stdout:1026: finishing write tid 4 to smithi017107884-23 2023-03-29T14:06:37.408 INFO:tasks.rados.rados.0.smithi017.stdout:1026: oid 23 updating version 0 to 209 2023-03-29T14:06:37.409 INFO:tasks.rados.rados.0.smithi017.stdout:1026: oid 23 version 209 is already newer than 209 2023-03-29T14:06:37.409 INFO:tasks.rados.rados.0.smithi017.stdout:1026: oid 23 version 209 is already newer than 209 2023-03-29T14:06:37.409 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 209 (ObjNum 509 snap 114 seq_num 509) dirty exists 2023-03-29T14:06:37.409 INFO:tasks.rados.rados.0.smithi017.stdout:1026: left oid 23 (ObjNum 509 snap 114 seq_num 509) 2023-03-29T14:06:38.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:38.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~88s 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1026: done (1 left) 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1027: done (0 left) 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1028: read oid 17 snap -1 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1028: expect deleted 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1029: read oid 50 snap -1 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1029: expect deleted 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1030: write oid 48 current snap is 115 2023-03-29T14:06:38.225 INFO:tasks.rados.rados.0.smithi017.stdout:1030: seq_num 510 ranges {571570=572641,1877346=694224} 2023-03-29T14:06:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:38.227+0000 7f7ad1576700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:38.231 INFO:tasks.rados.rados.0.smithi017.stdout:1030: writing smithi017107884-48 from 571570 to 1144211 tid 1 2023-03-29T14:06:38.232 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:38.232+0000 7f7ad1576700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1030: writing smithi017107884-48 from 1877346 to 2571570 tid 2 2023-03-29T14:06:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1031: write oid 30 current snap is 115 2023-03-29T14:06:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1031: seq_num 511 ranges {588377=647890,1935828=635840} 2023-03-29T14:06:38.247 INFO:tasks.rados.rados.0.smithi017.stdout:1031: writing smithi017107884-30 from 588377 to 1236267 tid 1 2023-03-29T14:06:38.254 INFO:tasks.rados.rados.0.smithi017.stdout:1031: writing smithi017107884-30 from 1935828 to 2571668 tid 2 2023-03-29T14:06:38.254 INFO:tasks.rados.rados.0.smithi017.stdout:1032: snap_remove snap 113 2023-03-29T14:06:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:38.473 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:38.474 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1032: done (4 left) 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1033: read oid 2 snap -1 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1033: expect (ObjNum 436 snap 99 seq_num 436) 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1034: read oid 29 snap -1 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1034: expect (ObjNum 429 snap 98 seq_num 429) 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1035: read oid 32 snap 112 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1035: expect deleted 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1036: delete oid 31 current snap is 115 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1031: finishing write tid 1 to smithi017107884-30 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1031: finishing write tid 2 to smithi017107884-30 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1031: finishing write tid 3 to smithi017107884-30 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1031: finishing write tid 4 to smithi017107884-30 2023-03-29T14:06:39.231 INFO:tasks.rados.rados.0.smithi017.stdout:1031: oid 30 updating version 0 to 53 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1031: oid 30 version 53 is already newer than 53 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1031: oid 30 version 53 is already newer than 53 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 53 (ObjNum 511 snap 115 seq_num 511) dirty exists 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1031: left oid 30 (ObjNum 511 snap 115 seq_num 511) 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: finishing write tid 1 to smithi017107884-48 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: finishing write tid 2 to smithi017107884-48 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: finishing write tid 3 to smithi017107884-48 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: finishing write tid 4 to smithi017107884-48 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: oid 48 updating version 0 to 149 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: oid 48 version 149 is already newer than 149 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: oid 48 version 149 is already newer than 149 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 149 (ObjNum 510 snap 115 seq_num 510) dirty exists 2023-03-29T14:06:39.232 INFO:tasks.rados.rados.0.smithi017.stdout:1030: left oid 48 (ObjNum 510 snap 115 seq_num 510) 2023-03-29T14:06:39.253 INFO:tasks.rados.rados.0.smithi017.stdout:1028: done (7 left) 2023-03-29T14:06:39.253 INFO:tasks.rados.rados.0.smithi017.stdout:1029: done (6 left) 2023-03-29T14:06:39.253 INFO:tasks.rados.rados.0.smithi017.stdout:1030: done (5 left) 2023-03-29T14:06:39.254 INFO:tasks.rados.rados.0.smithi017.stdout:1031: done (4 left) 2023-03-29T14:06:39.254 INFO:tasks.rados.rados.0.smithi017.stdout:1036: done (3 left) 2023-03-29T14:06:39.254 INFO:tasks.rados.rados.0.smithi017.stdout:1037: write oid 16 current snap is 115 2023-03-29T14:06:39.254 INFO:tasks.rados.rados.0.smithi017.stdout:1037: seq_num 513 ranges {621991=798388,1936439=639509} 2023-03-29T14:06:39.261 INFO:tasks.rados.rados.0.smithi017.stdout:1037: writing smithi017107884-16 from 621991 to 1420379 tid 1 2023-03-29T14:06:39.268 INFO:tasks.rados.rados.0.smithi017.stdout:1037: writing smithi017107884-16 from 1936439 to 2575948 tid 2 2023-03-29T14:06:39.268 INFO:tasks.rados.rados.0.smithi017.stdout:1038: read oid 13 snap -1 2023-03-29T14:06:39.268 INFO:tasks.rados.rados.0.smithi017.stdout:1038: expect (ObjNum 474 snap 109 seq_num 474) 2023-03-29T14:06:39.268 INFO:tasks.rados.rados.0.smithi017.stdout:1039: snap_remove snap 114 2023-03-29T14:06:40.231 INFO:tasks.rados.rados.0.smithi017.stdout:1039: done (5 left) 2023-03-29T14:06:40.331 INFO:tasks.rados.rados.0.smithi017.stdout:1040: write oid 45 current snap is 115 2023-03-29T14:06:40.331 INFO:tasks.rados.rados.0.smithi017.stdout:1037: finishing write tid 1 to smithi017107884-16 2023-03-29T14:06:40.331 INFO:tasks.rados.rados.0.smithi017.stdout:1040: seq_num 514 ranges {638798=473637,1594921=783167} 2023-03-29T14:06:40.332 INFO:tasks.rados.rados.0.smithi017.stdout:1040: writing smithi017107884-45 from 638798 to 1112435 tid 1 2023-03-29T14:06:40.332 INFO:tasks.rados.rados.0.smithi017.stdout:1040: writing smithi017107884-45 from 1594921 to 2378088 tid 2 2023-03-29T14:06:40.332 INFO:tasks.rados.rados.0.smithi017.stdout:1041: delete oid 14 current snap is 115 2023-03-29T14:06:40.332 INFO:tasks.rados.rados.0.smithi017.stdout:1038: expect (ObjNum 474 snap 109 seq_num 474) 2023-03-29T14:06:40.402 INFO:tasks.rados.rados.0.smithi017.stdout:1033: expect (ObjNum 436 snap 99 seq_num 436) 2023-03-29T14:06:40.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:40.532 INFO:tasks.rados.rados.0.smithi017.stdout:1037: finishing write tid 2 to smithi017107884-16 2023-03-29T14:06:40.532 INFO:tasks.rados.rados.0.smithi017.stdout:1033: done (6 left) 2023-03-29T14:06:40.532 INFO:tasks.rados.rados.0.smithi017.stdout:1038: done (5 left) 2023-03-29T14:06:40.532 INFO:tasks.rados.rados.0.smithi017.stdout:1041: done (4 left) 2023-03-29T14:06:40.532 INFO:tasks.rados.rados.0.smithi017.stdout:1042: write oid 26 current snap is 115 2023-03-29T14:06:40.532 INFO:tasks.rados.rados.0.smithi017.stdout:1034: expect (ObjNum 429 snap 98 seq_num 429) 2023-03-29T14:06:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:40.604+0000 7f4773c72700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:40.609+0000 7f4773c72700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:40.662 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:40.662 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:40.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:40.670 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:40.674 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:40.674 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:40.694 INFO:tasks.rados.rados.0.smithi017.stdout:1037: finishing write tid 3 to smithi017107884-16 2023-03-29T14:06:40.695 INFO:tasks.rados.rados.0.smithi017.stdout:1037: finishing write tid 4 to smithi017107884-16 2023-03-29T14:06:40.695 INFO:tasks.rados.rados.0.smithi017.stdout:1037: oid 16 updating version 0 to 70 2023-03-29T14:06:40.695 INFO:tasks.rados.rados.0.smithi017.stdout:1037: oid 16 version 70 is already newer than 70 2023-03-29T14:06:40.695 INFO:tasks.rados.rados.0.smithi017.stdout:1037: oid 16 version 70 is already newer than 70 2023-03-29T14:06:40.695 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 70 (ObjNum 513 snap 115 seq_num 513) dirty exists 2023-03-29T14:06:40.695 INFO:tasks.rados.rados.0.smithi017.stdout:1037: left oid 16 (ObjNum 513 snap 115 seq_num 513) 2023-03-29T14:06:40.695 INFO:tasks.rados.rados.0.smithi017.stdout:1042: seq_num 516 ranges {672412=624135,1995532=676880} 2023-03-29T14:06:40.703 INFO:tasks.rados.rados.0.smithi017.stdout:1042: writing smithi017107884-26 from 672412 to 1296547 tid 1 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1042: writing smithi017107884-26 from 1995532 to 2672412 tid 2 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1034: done (4 left) 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1035: done (3 left) 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1037: done (2 left) 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1043: write (excl) oid 5 current snap is 115 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1040: finishing write tid 1 to smithi017107884-45 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1040: finishing write tid 2 to smithi017107884-45 2023-03-29T14:06:40.714 INFO:tasks.rados.rados.0.smithi017.stdout:1043: seq_num 517 ranges {689219=415737,1770367=646847} 2023-03-29T14:06:40.721 INFO:tasks.rados.rados.0.smithi017.stdout:1043: writing smithi017107884-5 from 689219 to 1104956 tid 1 2023-03-29T14:06:40.732 INFO:tasks.rados.rados.0.smithi017.stdout:1043: writing smithi017107884-5 from 1770367 to 2417214 tid 2 2023-03-29T14:06:40.732 INFO:tasks.rados.rados.0.smithi017.stdout:1044: read oid 21 snap 112 2023-03-29T14:06:40.732 INFO:tasks.rados.rados.0.smithi017.stdout:1044: expect (ObjNum 498 snap 112 seq_num 498) 2023-03-29T14:06:40.732 INFO:tasks.rados.rados.0.smithi017.stdout:1045: delete oid 13 current snap is 115 2023-03-29T14:06:40.732 INFO:tasks.rados.rados.0.smithi017.stdout:1040: finishing write tid 3 to smithi017107884-45 2023-03-29T14:06:40.732 INFO:tasks.rados.rados.0.smithi017.stdout:1040: finishing write tid 4 to smithi017107884-45 2023-03-29T14:06:40.733 INFO:tasks.rados.rados.0.smithi017.stdout:1040: oid 45 updating version 0 to 157 2023-03-29T14:06:40.733 INFO:tasks.rados.rados.0.smithi017.stdout:1040: oid 45 version 157 is already newer than 157 2023-03-29T14:06:40.733 INFO:tasks.rados.rados.0.smithi017.stdout:1040: oid 45 version 157 is already newer than 157 2023-03-29T14:06:40.733 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 157 (ObjNum 514 snap 115 seq_num 514) dirty exists 2023-03-29T14:06:40.733 INFO:tasks.rados.rados.0.smithi017.stdout:1040: left oid 45 (ObjNum 514 snap 115 seq_num 514) 2023-03-29T14:06:40.745 INFO:tasks.rados.rados.0.smithi017.stdout:1042: finishing write tid 1 to smithi017107884-26 2023-03-29T14:06:40.769 INFO:tasks.rados.rados.0.smithi017.stdout:1042: finishing write tid 2 to smithi017107884-26 2023-03-29T14:06:40.788 INFO:tasks.rados.rados.0.smithi017.stdout:1042: finishing write tid 3 to smithi017107884-26 2023-03-29T14:06:40.790 INFO:tasks.rados.rados.0.smithi017.stdout:1042: finishing write tid 4 to smithi017107884-26 2023-03-29T14:06:40.791 INFO:tasks.rados.rados.0.smithi017.stdout:1042: oid 26 updating version 0 to 111 2023-03-29T14:06:40.791 INFO:tasks.rados.rados.0.smithi017.stdout:1042: oid 26 version 111 is already newer than 111 2023-03-29T14:06:40.791 INFO:tasks.rados.rados.0.smithi017.stdout:1042: oid 26 version 111 is already newer than 111 2023-03-29T14:06:40.791 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 111 (ObjNum 516 snap 115 seq_num 516) dirty exists 2023-03-29T14:06:40.791 INFO:tasks.rados.rados.0.smithi017.stdout:1042: left oid 26 (ObjNum 516 snap 115 seq_num 516) 2023-03-29T14:06:40.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:40.794+0000 7f1a4651f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:40.801 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:40.802+0000 7f1a4651f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:40.802 INFO:tasks.rados.rados.0.smithi017.stdout:1044: expect (ObjNum 498 snap 112 seq_num 498) 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: finishing write tid 1 to smithi017107884-5 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: finishing write tid 2 to smithi017107884-5 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: finishing write tid 3 to smithi017107884-5 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: finishing write tid 4 to smithi017107884-5 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: oid 5 updating version 0 to 155 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: oid 5 updating version 155 to 158 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: oid 5 version 158 is already newer than 158 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 158 (ObjNum 517 snap 115 seq_num 517) dirty exists 2023-03-29T14:06:40.901 INFO:tasks.rados.rados.0.smithi017.stdout:1043: left oid 5 (ObjNum 517 snap 115 seq_num 517) 2023-03-29T14:06:40.958 INFO:tasks.rados.rados.0.smithi017.stdout:1040: done (4 left) 2023-03-29T14:06:40.958 INFO:tasks.rados.rados.0.smithi017.stdout:1042: done (3 left) 2023-03-29T14:06:40.958 INFO:tasks.rados.rados.0.smithi017.stdout:1043: done (2 left) 2023-03-29T14:06:40.958 INFO:tasks.rados.rados.0.smithi017.stdout:1044: done (1 left) 2023-03-29T14:06:40.958 INFO:tasks.rados.rados.0.smithi017.stdout:1045: done (0 left) 2023-03-29T14:06:40.958 INFO:tasks.rados.rados.0.smithi017.stdout:1046: snap_create 2023-03-29T14:06:41.007 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:41.009 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:41.020 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2023-03-29T14:06:40.228923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":146431120,"num_objects":558,"num_object_clones":462,"num_object_copies":1674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":46,"num_read":2412,"num_read_kb":1349086,"num_write":3240,"num_write_kb":865342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1700,"ondisk_log_size":1700,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1126796,"kb_used_data":1063676,"kb_used_omap":0,"kb_used_meta":57920,"kb_avail":376360564,"statfs":{"total":386547056640,"available":385393217536,"internally_reserved":0,"allocated":1089204224,"data_stored":1304859162,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":59310080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31126564,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":70,"num_read_kb":44788,"num_write":96,"num_write_kb":22854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.946175"},"pg_stats":[{"pgid":"3.f","version":"222'66","reported_seq":133,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:35.297286+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:35.297286+0000","last_peered":"2023-03-29T14:06:35.297286+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:35.297286+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345022,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":21195,"num_write":52,"num_write_kb":14796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":4}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"240'29","reported_seq":261,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:34.369381+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:34.369381+0000","last_peered":"2023-03-29T14:06:34.369381+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:34.369381+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":44,"num_read_kb":25429,"num_write":54,"num_write_kb":16442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"248'145","reported_seq":526,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-03-29T14:06:35.284671+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:35.284671+0000","last_peered":"2023-03-29T14:06:35.284671+0000","last_clean":"2023-03-29T14:06:35.284671+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:35.284671+0000","last_undegraded":"2023-03-29T14:06:35.284671+0000","last_fullsized":"2023-03-29T14:06:35.284671+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12303054,"num_objects":50,"num_object_clones":40,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":210,"num_read_kb":136449,"num_write":272,"num_write_kb":79730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"250'149","reported_seq":540,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-03-29T14:06:36.408500+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:36.408500+0000","last_peered":"2023-03-29T14:06:36.408500+0000","last_clean":"2023-03-29T14:06:36.408500+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:36.408500+0000","last_undegraded":"2023-03-29T14:06:36.408500+0000","last_fullsized":"2023-03-29T14:06:36.408500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11328248,"num_objects":62,"num_object_clones":54,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":223,"num_read_kb":134490,"num_write":292,"num_write_kb":82024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"248'153","reported_seq":533,"reported_epoch":249,"state":"active+clean","last_fresh":"2023-03-29T14:06:34.655857+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:34.655857+0000","last_peered":"2023-03-29T14:06:34.655857+0000","last_clean":"2023-03-29T14:06:34.655857+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:34.655857+0000","last_undegraded":"2023-03-29T14:06:34.655857+0000","last_fullsized":"2023-03-29T14:06:34.655857+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13345352,"num_objects":56,"num_object_clones":48,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":197,"num_read_kb":110012,"num_write":310,"num_write_kb":89406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":207,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-03-29T14:06:35.291462+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:35.291462+0000","last_peered":"2023-03-29T14:06:35.291462+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:35.291462+0000","last_undegraded":"2023-03-29T14:06:35.291462+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"249'97","reported_seq":417,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:35.784071+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:35.784071+0000","last_peered":"2023-03-29T14:06:35.784071+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:35.784071+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975070,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":123,"num_read_kb":61238,"num_write":180,"num_write_kb":52872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"243'66","reported_seq":347,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-03-29T14:06:35.293187+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:35.293187+0000","last_peered":"2023-03-29T14:06:35.293187+0000","last_clean":"2023-03-29T14:06:35.293187+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:35.293187+0000","last_undegraded":"2023-03-29T14:06:35.293187+0000","last_fullsized":"2023-03-29T14:06:35.293187+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":87,"num_read_kb":49142,"num_write":126,"num_write_kb":35664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":200,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-03-29T14:06:34.363414+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:34.363414+0000","last_peered":"2023-03-29T14:06:34.363414+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:34.363414+0000","last_undegraded":"2023-03-29T14:06:34.363414+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"246'45","reported_seq":81,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-03-29T14:06:34.433444+0000","last_change":"2023-03-29T14:05:30.480618+0000","last_active":"2023-03-29T14:06:34.433444+0000","last_peered":"2023-03-29T14:06:34.433444+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:34.433444+0000","last_undegraded":"2023-03-29T14:06:34.433444+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2588542,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":12584,"num_write":16,"num_write_kb":3662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":210,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-03-29T14:06:35.296443+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:35.296443+0000","last_peered":"2023-03-29T14:06:35.296443+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:35.296443+0000","last_undegraded":"2023-03-29T14:06:35.296443+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"222'78","reported_seq":378,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:35.521262+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:35.521262+0000","last_peered":"2023-03-29T14:06:35.521262+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:35.521262+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":119,"num_read_kb":80399,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-03-29T14:06:34.341145+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:34.341145+0000","last_peered":"2023-03-29T14:06:34.341145+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:34.341145+0000","last_undegraded":"2023-03-29T14:06:34.341145+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"237'91","reported_seq":395,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:34.505654+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:34.505654+0000","last_peered":"2023-03-29T14:06:34.505654+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:34.505654+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":110,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":210,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-03-29T14:06:35.290310+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:35.290310+0000","last_peered":"2023-03-29T14:06:35.290310+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:35.290310+0000","last_undegraded":"2023-03-29T14:06:35.290310+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":425,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:34.375646+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:34.375646+0000","last_peered":"2023-03-29T14:06:34.375646+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:34.375646+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"249'107","reported_seq":489,"reported_epoch":250,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:35.348746+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:35.348746+0000","last_peered":"2023-03-29T14:06:35.348746+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:35.348746+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496582,"num_objects":36,"num_object_clones":28,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":228,"num_read_kb":150886,"num_write":214,"num_write_kb":61726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":36}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":210,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-03-29T14:06:35.282828+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:35.282828+0000","last_peered":"2023-03-29T14:06:35.282828+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:35.282828+0000","last_undegraded":"2023-03-29T14:06:35.282828+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"249'205","reported_seq":680,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:35.384911+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:35.384911+0000","last_peered":"2023-03-29T14:06:35.384911+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:35.384911+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22656430,"num_objects":70,"num_object_clones":56,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":357,"num_read_kb":220595,"num_write":380,"num_write_kb":109418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":200,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-03-29T14:06:34.367025+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:34.367025+0000","last_peered":"2023-03-29T14:06:34.367025+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:34.367025+0000","last_undegraded":"2023-03-29T14:06:34.367025+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"240'74","reported_seq":370,"reported_epoch":249,"state":"active+clean","last_fresh":"2023-03-29T14:06:35.625451+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:35.625451+0000","last_peered":"2023-03-29T14:06:35.625451+0000","last_clean":"2023-03-29T14:06:35.625451+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:35.625451+0000","last_undegraded":"2023-03-29T14:06:35.625451+0000","last_fullsized":"2023-03-29T14:06:35.625451+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4470794,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":70176,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"249'153","reported_seq":530,"reported_epoch":249,"state":"active+clean","last_fresh":"2023-03-29T14:06:35.576278+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:35.576278+0000","last_peered":"2023-03-29T14:06:35.576278+0000","last_clean":"2023-03-29T14:06:35.576278+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:35.576278+0000","last_undegraded":"2023-03-29T14:06:35.576278+0000","last_fullsized":"2023-03-29T14:06:35.576278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22286742,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":188,"num_read_kb":105971,"num_write":298,"num_write_kb":87566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":200,"reported_epoch":249,"state":"active+undersized","last_fresh":"2023-03-29T14:06:34.383025+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:34.383025+0000","last_peered":"2023-03-29T14:06:34.383025+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:34.383025+0000","last_undegraded":"2023-03-29T14:06:34.383025+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":210,"reported_epoch":250,"state":"active+undersized","last_fresh":"2023-03-29T14:06:35.293714+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:35.293714+0000","last_peered":"2023-03-29T14:06:35.293714+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:35.293714+0000","last_undegraded":"2023-03-29T14:06:35.293714+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"249'145","reported_seq":520,"reported_epoch":249,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:35.508206+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:35.508206+0000","last_peered":"2023-03-29T14:06:35.508206+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:35.508206+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12034142,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":200,"num_read_kb":119527,"num_write":282,"num_write_kb":85334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":145250384,"num_objects":554,"num_object_clones":462,"num_object_copies":1662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":46,"num_read":2270,"num_read_kb":1348966,"num_write":2966,"num_write_kb":861554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1603,"ondisk_log_size":1603,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":300308,"kb_used_data":282216,"kb_used_omap":0,"kb_used_meta":14656,"kb_avail":94071532,"statfs":{"total":96636764160,"available":96329248768,"internally_reserved":0,"allocated":288989184,"data_stored":342572911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15007744},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":346756,"kb_used_data":330308,"kb_used_omap":0,"kb_used_meta":15744,"kb_avail":94025084,"statfs":{"total":96636764160,"available":96281686016,"internally_reserved":0,"allocated":338235392,"data_stored":408198518,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16121856},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":323788,"kb_used_data":307312,"kb_used_omap":0,"kb_used_meta":15424,"kb_avail":94048052,"statfs":{"total":96636764160,"available":96305205248,"internally_reserved":0,"allocated":314687488,"data_stored":380855508,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":15794176},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:41.297 INFO:tasks.rados.rados.0.smithi017.stdout:1046: done (0 left) 2023-03-29T14:06:41.297 INFO:tasks.rados.rados.0.smithi017.stdout:1047: read oid 48 snap -1 2023-03-29T14:06:41.297 INFO:tasks.rados.rados.0.smithi017.stdout:1047: expect (ObjNum 510 snap 115 seq_num 510) 2023-03-29T14:06:41.297 INFO:tasks.rados.rados.0.smithi017.stdout:1048: write (excl) oid 45 current snap is 116 2023-03-29T14:06:41.297 INFO:tasks.rados.rados.0.smithi017.stdout:1048: seq_num 519 ranges {722833=566235,1887331=650516} 2023-03-29T14:06:41.306 INFO:tasks.rados.rados.0.smithi017.stdout:1048: writing smithi017107884-45 from 722833 to 1289068 tid 1 2023-03-29T14:06:41.315 INFO:tasks.rados.rados.0.smithi017.stdout:1048: writing smithi017107884-45 from 1887331 to 2537847 tid 2 2023-03-29T14:06:41.315 INFO:tasks.rados.rados.0.smithi017.stdout:1049: delete oid 9 current snap is 116 2023-03-29T14:06:41.348 INFO:tasks.rados.rados.0.smithi017.stdout:1047: expect (ObjNum 510 snap 115 seq_num 510) 2023-03-29T14:06:41.506 INFO:tasks.rados.rados.0.smithi017.stdout:1048: finishing write tid 1 to smithi017107884-45 2023-03-29T14:06:41.507 INFO:tasks.rados.rados.0.smithi017.stdout:1048: finishing write tid 2 to smithi017107884-45 2023-03-29T14:06:41.507 INFO:tasks.rados.rados.0.smithi017.stdout:1047: done (2 left) 2023-03-29T14:06:41.507 INFO:tasks.rados.rados.0.smithi017.stdout:1049: done (1 left) 2023-03-29T14:06:41.507 INFO:tasks.rados.rados.0.smithi017.stdout:1050: read oid 46 snap -1 2023-03-29T14:06:41.507 INFO:tasks.rados.rados.0.smithi017.stdout:1050: expect deleted 2023-03-29T14:06:41.507 INFO:tasks.rados.rados.0.smithi017.stdout:1051: write oid 7 current snap is 116 2023-03-29T14:06:41.507 INFO:tasks.rados.rados.0.smithi017.stdout:1051: seq_num 521 ranges {756447=716733,1887942=537832} 2023-03-29T14:06:41.514 INFO:tasks.rados.rados.0.smithi017.stdout:1051: writing smithi017107884-7 from 756447 to 1473180 tid 1 2023-03-29T14:06:41.520 INFO:tasks.rados.rados.0.smithi017.stdout:1051: writing smithi017107884-7 from 1887942 to 2425774 tid 2 2023-03-29T14:06:41.520 INFO:tasks.rados.rados.0.smithi017.stdout:1052: read oid 50 snap 115 2023-03-29T14:06:41.521 INFO:tasks.rados.rados.0.smithi017.stdout:1052: expect deleted 2023-03-29T14:06:41.521 INFO:tasks.rados.rados.0.smithi017.stdout:1053: delete oid 47 current snap is 116 2023-03-29T14:06:41.532 INFO:tasks.rados.rados.0.smithi017.stdout:1051: finishing write tid 1 to smithi017107884-7 2023-03-29T14:06:41.538 INFO:tasks.rados.rados.0.smithi017.stdout:1048: finishing write tid 3 to smithi017107884-45 2023-03-29T14:06:41.554 INFO:tasks.rados.rados.0.smithi017.stdout:1051: finishing write tid 2 to smithi017107884-7 2023-03-29T14:06:41.572 INFO:tasks.rados.rados.0.smithi017.stdout:1051: finishing write tid 3 to smithi017107884-7 2023-03-29T14:06:41.573 INFO:tasks.rados.rados.0.smithi017.stdout:1051: finishing write tid 4 to smithi017107884-7 2023-03-29T14:06:41.573 INFO:tasks.rados.rados.0.smithi017.stdout:1051: oid 7 updating version 0 to 69 2023-03-29T14:06:41.573 INFO:tasks.rados.rados.0.smithi017.stdout:1051: oid 7 version 69 is already newer than 69 2023-03-29T14:06:41.573 INFO:tasks.rados.rados.0.smithi017.stdout:1051: oid 7 version 69 is already newer than 69 2023-03-29T14:06:41.573 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 69 (ObjNum 521 snap 116 seq_num 521) dirty exists 2023-03-29T14:06:41.573 INFO:tasks.rados.rados.0.smithi017.stdout:1051: left oid 7 (ObjNum 521 snap 116 seq_num 521) 2023-03-29T14:06:41.579 INFO:tasks.rados.rados.0.smithi017.stdout:1048: finishing write tid 4 to smithi017107884-45 2023-03-29T14:06:41.579 INFO:tasks.rados.rados.0.smithi017.stdout:1048: oid 45 updating version 0 to 161 2023-03-29T14:06:41.579 INFO:tasks.rados.rados.0.smithi017.stdout:1048: oid 45 version 161 is already newer than 161 2023-03-29T14:06:41.579 INFO:tasks.rados.rados.0.smithi017.stdout:1048: oid 45 version 161 is already newer than 158 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 161 (ObjNum 519 snap 116 seq_num 519) dirty exists 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:1048: left oid 45 (ObjNum 519 snap 116 seq_num 519) 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:1048: done (4 left) 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:1050: done (3 left) 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:1051: done (2 left) 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:1052: done (1 left) 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:1053: done (0 left) 2023-03-29T14:06:41.580 INFO:tasks.rados.rados.0.smithi017.stdout:1054: snap_create 2023-03-29T14:06:42.302 INFO:tasks.rados.rados.0.smithi017.stdout:1054: done (0 left) 2023-03-29T14:06:42.302 INFO:tasks.rados.rados.0.smithi017.stdout:1055: write (excl) oid 38 current snap is 117 2023-03-29T14:06:42.302 INFO:tasks.rados.rados.0.smithi017.stdout:1055: seq_num 523 ranges {790061=467231,2004906=541501} 2023-03-29T14:06:42.309 INFO:tasks.rados.rados.0.smithi017.stdout:1055: writing smithi017107884-38 from 790061 to 1257292 tid 1 2023-03-29T14:06:42.316 INFO:tasks.rados.rados.0.smithi017.stdout:1055: writing smithi017107884-38 from 2004906 to 2546407 tid 2 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1056: read oid 25 snap -1 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1056: expect (ObjNum 507 snap 113 seq_num 507) 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1057: read oid 17 snap -1 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1057: expect deleted 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1058: read oid 43 snap -1 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1058: expect deleted 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1059: read oid 18 snap 115 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1059: expect deleted 2023-03-29T14:06:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1060: snap_create 2023-03-29T14:06:42.342 INFO:tasks.rados.rados.0.smithi017.stdout:1056: expect (ObjNum 507 snap 113 seq_num 507) 2023-03-29T14:06:42.518 INFO:tasks.rados.rados.0.smithi017.stdout:1055: finishing write tid 1 to smithi017107884-38 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1055: finishing write tid 2 to smithi017107884-38 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1055: finishing write tid 3 to smithi017107884-38 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1055: finishing write tid 4 to smithi017107884-38 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1055: oid 38 updating version 0 to 32 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1055: oid 38 version 32 is already newer than 30 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1055: oid 38 version 32 is already newer than 32 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 32 (ObjNum 523 snap 117 seq_num 523) dirty exists 2023-03-29T14:06:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1055: left oid 38 (ObjNum 523 snap 117 seq_num 523) 2023-03-29T14:06:42.675 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:42.804 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:42.803+0000 7fee8b7e0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:42.808 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:42.808+0000 7fee8b7e0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:42.869 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:42.872 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:42.872 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:43.303 INFO:tasks.rados.rados.0.smithi017.stdout:1055: done (5 left) 2023-03-29T14:06:43.304 INFO:tasks.rados.rados.0.smithi017.stdout:1056: done (4 left) 2023-03-29T14:06:43.304 INFO:tasks.rados.rados.0.smithi017.stdout:1057: done (3 left) 2023-03-29T14:06:43.304 INFO:tasks.rados.rados.0.smithi017.stdout:1058: done (2 left) 2023-03-29T14:06:43.304 INFO:tasks.rados.rados.0.smithi017.stdout:1059: done (1 left) 2023-03-29T14:06:43.304 INFO:tasks.rados.rados.0.smithi017.stdout:1060: done (0 left) 2023-03-29T14:06:43.304 INFO:tasks.rados.rados.0.smithi017.stdout:1061: snap_create 2023-03-29T14:06:43.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~94s 2023-03-29T14:06:44.312 INFO:tasks.rados.rados.0.smithi017.stdout:1061: done (0 left) 2023-03-29T14:06:44.312 INFO:tasks.rados.rados.0.smithi017.stdout:1062: read oid 24 snap 115 2023-03-29T14:06:44.312 INFO:tasks.rados.rados.0.smithi017.stdout:1062: expect deleted 2023-03-29T14:06:44.312 INFO:tasks.rados.rados.0.smithi017.stdout:1063: snap_create 2023-03-29T14:06:44.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:45.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:45.001+0000 7fa027fad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:45.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:45.005+0000 7fa027fad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:45.081 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:45.082 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:45.313 INFO:tasks.rados.rados.0.smithi017.stdout:1062: done (1 left) 2023-03-29T14:06:45.313 INFO:tasks.rados.rados.0.smithi017.stdout:1063: done (0 left) 2023-03-29T14:06:45.314 INFO:tasks.rados.rados.0.smithi017.stdout:1064: write (excl) oid 37 current snap is 120 2023-03-29T14:06:45.314 INFO:tasks.rados.rados.0.smithi017.stdout:1064: seq_num 524 ranges {406868=542480,1547035=401512,2690354=116514} 2023-03-29T14:06:45.318 INFO:tasks.rados.rados.0.smithi017.stdout:1064: writing smithi017107884-37 from 406868 to 949348 tid 1 2023-03-29T14:06:45.323 INFO:tasks.rados.rados.0.smithi017.stdout:1064: writing smithi017107884-37 from 1547035 to 1948547 tid 2 2023-03-29T14:06:45.329 INFO:tasks.rados.rados.0.smithi017.stdout:1064: writing smithi017107884-37 from 2690354 to 2806868 tid 3 2023-03-29T14:06:45.329 INFO:tasks.rados.rados.0.smithi017.stdout:1065: read oid 22 snap -1 2023-03-29T14:06:45.329 INFO:tasks.rados.rados.0.smithi017.stdout:1065: expect (ObjNum 434 snap 99 seq_num 434) 2023-03-29T14:06:45.329 INFO:tasks.rados.rados.0.smithi017.stdout:1066: snap_remove snap 116 2023-03-29T14:06:46.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:46.025 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:46.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:46.153+0000 7fda183fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:46.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:46.160+0000 7fda183fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:46.323 INFO:tasks.rados.rados.0.smithi017.stdout:1066: done (2 left) 2023-03-29T14:06:46.323 INFO:tasks.rados.rados.0.smithi017.stdout:1067: read oid 15 snap -1 2023-03-29T14:06:46.323 INFO:tasks.rados.rados.0.smithi017.stdout:1067: expect (ObjNum 373 snap 90 seq_num 373) 2023-03-29T14:06:46.323 INFO:tasks.rados.rados.0.smithi017.stdout:1068: write (excl) oid 16 current snap is 120 2023-03-29T14:06:46.323 INFO:tasks.rados.rados.0.smithi017.stdout:1068: seq_num 525 ranges {423675=734082,1721870=545170,2717777=105898} 2023-03-29T14:06:46.329 INFO:tasks.rados.rados.0.smithi017.stdout:1068: writing smithi017107884-16 from 423675 to 1157757 tid 1 2023-03-29T14:06:46.335 INFO:tasks.rados.rados.0.smithi017.stdout:1068: writing smithi017107884-16 from 1721870 to 2267040 tid 2 2023-03-29T14:06:46.339 INFO:tasks.rados.rados.0.smithi017.stdout:1068: writing smithi017107884-16 from 2717777 to 2823675 tid 3 2023-03-29T14:06:46.339 INFO:tasks.rados.rados.0.smithi017.stdout:1069: read oid 6 snap -1 2023-03-29T14:06:46.339 INFO:tasks.rados.rados.0.smithi017.stdout:1069: expect deleted 2023-03-29T14:06:46.339 INFO:tasks.rados.rados.0.smithi017.stdout:1070: write (excl) oid 30 current snap is 120 2023-03-29T14:06:46.339 INFO:tasks.rados.rados.0.smithi017.stdout:1070: seq_num 526 ranges {440482=409331,1380352=405181,2461553=378929} 2023-03-29T14:06:46.343 INFO:tasks.rados.rados.0.smithi017.stdout:1070: writing smithi017107884-30 from 440482 to 849813 tid 1 2023-03-29T14:06:46.348 INFO:tasks.rados.rados.0.smithi017.stdout:1070: writing smithi017107884-30 from 1380352 to 1785533 tid 2 2023-03-29T14:06:46.354 INFO:tasks.rados.rados.0.smithi017.stdout:1070: writing smithi017107884-30 from 2461553 to 2840482 tid 3 2023-03-29T14:06:46.354 INFO:tasks.rados.rados.0.smithi017.stdout:1071: write oid 23 current snap is 120 2023-03-29T14:06:46.354 INFO:tasks.rados.rados.0.smithi017.stdout:1065: expect (ObjNum 434 snap 99 seq_num 434) 2023-03-29T14:06:46.366 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:46.369 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:46.385 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2023-03-29T14:06:46.320110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134195888,"num_objects":584,"num_object_clones":488,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":48,"num_read":2472,"num_read_kb":1385506,"num_write":3338,"num_write_kb":894388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1754,"ondisk_log_size":1754,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1145284,"kb_used_data":1086564,"kb_used_omap":0,"kb_used_meta":58688,"kb_avail":376342076,"statfs":{"total":386547056640,"available":385374285824,"internally_reserved":0,"allocated":1112641536,"data_stored":1336003798,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":60096512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6286016,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":9,"num_read_kb":3,"num_write":26,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020282"},"pg_stats":[{"pgid":"3.f","version":"253'70","reported_seq":145,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:40.270338+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:40.270338+0000","last_peered":"2023-03-29T14:06:40.270338+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:40.270338+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11832458,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":21196,"num_write":60,"num_write_kb":17606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":6}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"256'32","reported_seq":274,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:42.495993+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:42.495993+0000","last_peered":"2023-03-29T14:06:42.495993+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:42.495993+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5580122,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":25430,"num_write":62,"num_write_kb":18414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"252'149","reported_seq":541,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-03-29T14:06:41.344113+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:41.344113+0000","last_peered":"2023-03-29T14:06:41.344113+0000","last_clean":"2023-03-29T14:06:41.344113+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:41.344113+0000","last_undegraded":"2023-03-29T14:06:41.344113+0000","last_fullsized":"2023-03-29T14:06:41.344113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748456,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":218,"num_read_kb":143078,"num_write":280,"num_write_kb":82206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"250'149","reported_seq":547,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-03-29T14:06:41.330950+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:41.330950+0000","last_peered":"2023-03-29T14:06:41.330950+0000","last_clean":"2023-03-29T14:06:41.330950+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:41.330950+0000","last_undegraded":"2023-03-29T14:06:41.330950+0000","last_fullsized":"2023-03-29T14:06:41.330950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5042298,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":223,"num_read_kb":134490,"num_write":292,"num_write_kb":82024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"255'161","reported_seq":553,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-03-29T14:06:42.341463+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:42.341463+0000","last_peered":"2023-03-29T14:06:42.341463+0000","last_clean":"2023-03-29T14:06:42.341463+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:42.341463+0000","last_undegraded":"2023-03-29T14:06:42.341463+0000","last_fullsized":"2023-03-29T14:06:42.341463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252930,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":201,"num_read_kb":110014,"num_write":326,"num_write_kb":94240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":212,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-03-29T14:06:40.269005+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:40.269005+0000","last_peered":"2023-03-29T14:06:40.269005+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:40.269005+0000","last_undegraded":"2023-03-29T14:06:40.269005+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"254'100","reported_seq":425,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:40.273585+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:40.273585+0000","last_peered":"2023-03-29T14:06:40.273585+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:40.273585+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":126,"num_read_kb":61238,"num_write":180,"num_write_kb":52872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'68","reported_seq":358,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-03-29T14:06:41.529024+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:41.529024+0000","last_peered":"2023-03-29T14:06:41.529024+0000","last_clean":"2023-03-29T14:06:41.529024+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:41.529024+0000","last_undegraded":"2023-03-29T14:06:41.529024+0000","last_fullsized":"2023-03-29T14:06:41.529024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":87,"num_read_kb":49142,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.352572+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:42.352572+0000","last_peered":"2023-03-29T14:06:42.352572+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:42.352572+0000","last_undegraded":"2023-03-29T14:06:42.352572+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"259'58","reported_seq":113,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:45.454673+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:06:45.454673+0000","last_peered":"2023-03-29T14:06:45.454673+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:45.454673+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706290,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":12587,"num_write":42,"num_write_kb":11134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-03-29T14:06:40.259843+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:40.259843+0000","last_peered":"2023-03-29T14:06:40.259843+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:40.259843+0000","last_undegraded":"2023-03-29T14:06:40.259843+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":390,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:44.387638+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:44.387638+0000","last_peered":"2023-03-29T14:06:44.387638+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:44.387638+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.364703+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:42.364703+0000","last_peered":"2023-03-29T14:06:42.364703+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:42.364703+0000","last_undegraded":"2023-03-29T14:06:42.364703+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"237'91","reported_seq":401,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:42.374414+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:42.374414+0000","last_peered":"2023-03-29T14:06:42.374414+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:42.374414+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":110,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":215,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-03-29T14:06:40.259107+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:40.259107+0000","last_peered":"2023-03-29T14:06:40.259107+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:40.259107+0000","last_undegraded":"2023-03-29T14:06:40.259107+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":431,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:42.354547+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:42.354547+0000","last_peered":"2023-03-29T14:06:42.354547+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:42.354547+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"254'111","reported_seq":504,"reported_epoch":255,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:40.799957+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:40.799957+0000","last_peered":"2023-03-29T14:06:40.799957+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:40.799957+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17479808,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":236,"num_read_kb":156411,"num_write":222,"num_write_kb":64270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":215,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-03-29T14:06:40.263996+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:40.263996+0000","last_peered":"2023-03-29T14:06:40.263996+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:40.263996+0000","last_undegraded":"2023-03-29T14:06:40.263996+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"251'209","reported_seq":699,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:44.390637+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:44.390637+0000","last_peered":"2023-03-29T14:06:44.390637+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:44.390637+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975796,"num_objects":72,"num_object_clones":58,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":365,"num_read_kb":227364,"num_write":388,"num_write_kb":111828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":72}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":206,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.356526+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:42.356526+0000","last_peered":"2023-03-29T14:06:42.356526+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:42.356526+0000","last_undegraded":"2023-03-29T14:06:42.356526+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"249'75","reported_seq":376,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-03-29T14:06:42.360696+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:42.360696+0000","last_peered":"2023-03-29T14:06:42.360696+0000","last_clean":"2023-03-29T14:06:42.360696+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:42.360696+0000","last_undegraded":"2023-03-29T14:06:42.360696+0000","last_fullsized":"2023-03-29T14:06:42.360696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4470794,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":70176,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"254'160","reported_seq":549,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-03-29T14:06:42.336613+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:42.336613+0000","last_peered":"2023-03-29T14:06:42.336613+0000","last_clean":"2023-03-29T14:06:42.336613+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:42.336613+0000","last_undegraded":"2023-03-29T14:06:42.336613+0000","last_fullsized":"2023-03-29T14:06:42.336613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958758,"num_objects":56,"num_object_clones":46,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":200,"num_read_kb":115324,"num_write":306,"num_write_kb":89642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":206,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.362324+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:42.362324+0000","last_peered":"2023-03-29T14:06:42.362324+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:42.362324+0000","last_undegraded":"2023-03-29T14:06:42.362324+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":215,"reported_epoch":255,"state":"active+undersized","last_fresh":"2023-03-29T14:06:40.266960+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:40.266960+0000","last_peered":"2023-03-29T14:06:40.266960+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:40.266960+0000","last_undegraded":"2023-03-29T14:06:40.266960+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"249'145","reported_seq":530,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:44.371658+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:44.371658+0000","last_peered":"2023-03-29T14:06:44.371658+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:44.371658+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12034142,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":201,"num_read_kb":119527,"num_write":282,"num_write_kb":85334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":133015152,"num_objects":580,"num_object_clones":488,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":48,"num_read":2330,"num_read_kb":1385386,"num_write":3064,"num_write_kb":890600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1657,"ondisk_log_size":1657,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308464,"kb_used_data":293608,"kb_used_omap":0,"kb_used_meta":14848,"kb_avail":94063376,"statfs":{"total":96636764160,"available":96320897024,"internally_reserved":0,"allocated":300654592,"data_stored":357642816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15204352},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":354636,"kb_used_data":338564,"kb_used_omap":0,"kb_used_meta":16064,"kb_avail":94017204,"statfs":{"total":96636764160,"available":96273616896,"internally_reserved":0,"allocated":346689536,"data_stored":419265414,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16449536},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":326240,"kb_used_data":310552,"kb_used_omap":0,"kb_used_meta":15680,"kb_avail":94045600,"statfs":{"total":96636764160,"available":96302694400,"internally_reserved":0,"allocated":318005248,"data_stored":385863343,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16056320},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:46.526 INFO:tasks.rados.rados.0.smithi017.stdout:1064: finishing write tid 1 to smithi017107884-37 2023-03-29T14:06:46.526 INFO:tasks.rados.rados.0.smithi017.stdout:1064: finishing write tid 2 to smithi017107884-37 2023-03-29T14:06:46.526 INFO:tasks.rados.rados.0.smithi017.stdout:1064: finishing write tid 3 to smithi017107884-37 2023-03-29T14:06:46.526 INFO:tasks.rados.rados.0.smithi017.stdout:1064: finishing write tid 4 to smithi017107884-37 2023-03-29T14:06:46.526 INFO:tasks.rados.rados.0.smithi017.stdout:1064: finishing write tid 5 to smithi017107884-37 2023-03-29T14:06:46.526 INFO:tasks.rados.rados.0.smithi017.stdout:1064: oid 37 updating version 0 to 58 2023-03-29T14:06:46.527 INFO:tasks.rados.rados.0.smithi017.stdout:1064: oid 37 version 58 is already newer than 58 2023-03-29T14:06:46.527 INFO:tasks.rados.rados.0.smithi017.stdout:1064: oid 37 version 58 is already newer than 58 2023-03-29T14:06:46.527 INFO:tasks.rados.rados.0.smithi017.stdout:1064: oid 37 version 58 is already newer than 54 2023-03-29T14:06:46.527 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 58 (ObjNum 524 snap 120 seq_num 524) dirty exists 2023-03-29T14:06:46.527 INFO:tasks.rados.rados.0.smithi017.stdout:1064: left oid 37 (ObjNum 524 snap 120 seq_num 524) 2023-03-29T14:06:46.527 INFO:tasks.rados.rados.0.smithi017.stdout:1071: seq_num 527 ranges {457289=484580,1438834=548839,2772623=84666} 2023-03-29T14:06:46.531 INFO:tasks.rados.rados.0.smithi017.stdout:1071: writing smithi017107884-23 from 457289 to 941869 tid 1 2023-03-29T14:06:46.537 INFO:tasks.rados.rados.0.smithi017.stdout:1071: writing smithi017107884-23 from 1438834 to 1987673 tid 2 2023-03-29T14:06:46.542 INFO:tasks.rados.rados.0.smithi017.stdout:1071: writing smithi017107884-23 from 2772623 to 2857289 tid 3 2023-03-29T14:06:46.542 INFO:tasks.rados.rados.0.smithi017.stdout:1064: done (6 left) 2023-03-29T14:06:46.542 INFO:tasks.rados.rados.0.smithi017.stdout:1065: done (5 left) 2023-03-29T14:06:46.542 INFO:tasks.rados.rados.0.smithi017.stdout:1072: write oid 18 current snap is 120 2023-03-29T14:06:46.542 INFO:tasks.rados.rados.0.smithi017.stdout:1068: finishing write tid 1 to smithi017107884-16 2023-03-29T14:06:46.542 INFO:tasks.rados.rados.0.smithi017.stdout:1067: expect (ObjNum 373 snap 90 seq_num 373) 2023-03-29T14:06:46.665 INFO:tasks.rados.rados.0.smithi017.stdout:1070: finishing write tid 1 to smithi017107884-30 2023-03-29T14:06:46.665 INFO:tasks.rados.rados.0.smithi017.stdout:1068: finishing write tid 2 to smithi017107884-16 2023-03-29T14:06:46.665 INFO:tasks.rados.rados.0.smithi017.stdout:1068: finishing write tid 3 to smithi017107884-16 2023-03-29T14:06:46.665 INFO:tasks.rados.rados.0.smithi017.stdout:1068: finishing write tid 4 to smithi017107884-16 2023-03-29T14:06:46.665 INFO:tasks.rados.rados.0.smithi017.stdout:1072: seq_num 528 ranges {474096=559829,1780963=408850,2800046=74050} 2023-03-29T14:06:46.670 INFO:tasks.rados.rados.0.smithi017.stdout:1072: writing smithi017107884-18 from 474096 to 1033925 tid 1 2023-03-29T14:06:46.677 INFO:tasks.rados.rados.0.smithi017.stdout:1072: writing smithi017107884-18 from 1780963 to 2189813 tid 2 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1072: writing smithi017107884-18 from 2800046 to 2874096 tid 3 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1067: done (5 left) 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1069: done (4 left) 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1073: write oid 22 current snap is 120 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1068: finishing write tid 5 to smithi017107884-16 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1068: oid 16 updating version 0 to 75 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1068: oid 16 version 75 is already newer than 75 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1068: oid 16 version 75 is already newer than 75 2023-03-29T14:06:46.681 INFO:tasks.rados.rados.0.smithi017.stdout:1068: oid 16 version 75 is already newer than 71 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 75 (ObjNum 525 snap 120 seq_num 525) dirty exists 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1068: left oid 16 (ObjNum 525 snap 120 seq_num 525) 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: finishing write tid 2 to smithi017107884-30 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: finishing write tid 3 to smithi017107884-30 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: finishing write tid 4 to smithi017107884-30 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: finishing write tid 5 to smithi017107884-30 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: oid 30 updating version 0 to 63 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: oid 30 version 63 is already newer than 59 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: oid 30 version 63 is already newer than 63 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: oid 30 version 63 is already newer than 63 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 63 (ObjNum 526 snap 120 seq_num 526) dirty exists 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1070: left oid 30 (ObjNum 526 snap 120 seq_num 526) 2023-03-29T14:06:46.682 INFO:tasks.rados.rados.0.smithi017.stdout:1073: seq_num 529 ranges {490903=635078,1839445=552508} 2023-03-29T14:06:46.687 INFO:tasks.rados.rados.0.smithi017.stdout:1073: writing smithi017107884-22 from 490903 to 1125981 tid 1 2023-03-29T14:06:46.694 INFO:tasks.rados.rados.0.smithi017.stdout:1073: writing smithi017107884-22 from 1839445 to 2391953 tid 2 2023-03-29T14:06:46.694 INFO:tasks.rados.rados.0.smithi017.stdout:1068: done (4 left) 2023-03-29T14:06:46.694 INFO:tasks.rados.rados.0.smithi017.stdout:1070: done (3 left) 2023-03-29T14:06:46.694 INFO:tasks.rados.rados.0.smithi017.stdout:1074: snap_remove snap 118 2023-03-29T14:06:47.083 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:47.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:47.211+0000 7fdc31ee7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:47.217+0000 7fdc31ee7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:47.281 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:47.284 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:47.284 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:47.324 INFO:tasks.rados.rados.0.smithi017.stdout:1074: done (3 left) 2023-03-29T14:06:47.324 INFO:tasks.rados.rados.0.smithi017.stdout:1075: write oid 6 current snap is 120 2023-03-29T14:06:47.324 INFO:tasks.rados.rados.0.smithi017.stdout:1071: finishing write tid 1 to smithi017107884-23 2023-03-29T14:06:47.324 INFO:tasks.rados.rados.0.smithi017.stdout:1075: seq_num 530 ranges {507710=710327,1897927=412519,2854892=52818} 2023-03-29T14:06:47.330 INFO:tasks.rados.rados.0.smithi017.stdout:1075: writing smithi017107884-6 from 507710 to 1218037 tid 1 2023-03-29T14:06:47.336 INFO:tasks.rados.rados.0.smithi017.stdout:1075: writing smithi017107884-6 from 1897927 to 2310446 tid 2 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1075: writing smithi017107884-6 from 2854892 to 2907710 tid 3 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1076: write (excl) oid 32 current snap is 120 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: finishing write tid 2 to smithi017107884-23 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: finishing write tid 3 to smithi017107884-23 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: finishing write tid 4 to smithi017107884-23 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: finishing write tid 5 to smithi017107884-23 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: oid 23 updating version 0 to 212 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: oid 23 updating version 212 to 214 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: oid 23 version 214 is already newer than 214 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: oid 23 version 214 is already newer than 214 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 214 (ObjNum 527 snap 120 seq_num 527) dirty exists 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1071: left oid 23 (ObjNum 527 snap 120 seq_num 527) 2023-03-29T14:06:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1076: seq_num 531 ranges {524517=785576,1956409=556177} 2023-03-29T14:06:47.347 INFO:tasks.rados.rados.0.smithi017.stdout:1076: writing smithi017107884-32 from 524517 to 1310093 tid 1 2023-03-29T14:06:47.354 INFO:tasks.rados.rados.0.smithi017.stdout:1076: writing smithi017107884-32 from 1956409 to 2512586 tid 2 2023-03-29T14:06:47.354 INFO:tasks.rados.rados.0.smithi017.stdout:1071: done (4 left) 2023-03-29T14:06:47.354 INFO:tasks.rados.rados.0.smithi017.stdout:1077: read oid 27 snap -1 2023-03-29T14:06:47.354 INFO:tasks.rados.rados.0.smithi017.stdout:1077: expect (ObjNum 458 snap 103 seq_num 458) 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: finishing write tid 1 to smithi017107884-18 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1078: read oid 28 snap -1 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1078: expect deleted 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1079: delete oid 41 current snap is 120 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: finishing write tid 2 to smithi017107884-18 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: finishing write tid 3 to smithi017107884-18 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: finishing write tid 4 to smithi017107884-18 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: finishing write tid 5 to smithi017107884-18 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: oid 18 updating version 0 to 104 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: oid 18 version 104 is already newer than 104 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: oid 18 version 104 is already newer than 103 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: oid 18 version 104 is already newer than 104 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 104 (ObjNum 528 snap 120 seq_num 528) dirty exists 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1072: left oid 18 (ObjNum 528 snap 120 seq_num 528) 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1073: finishing write tid 1 to smithi017107884-22 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1073: finishing write tid 2 to smithi017107884-22 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1073: finishing write tid 3 to smithi017107884-22 2023-03-29T14:06:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1073: finishing write tid 4 to smithi017107884-22 2023-03-29T14:06:47.356 INFO:tasks.rados.rados.0.smithi017.stdout:1073: oid 22 updating version 0 to 79 2023-03-29T14:06:47.356 INFO:tasks.rados.rados.0.smithi017.stdout:1073: oid 22 version 79 is already newer than 79 2023-03-29T14:06:47.356 INFO:tasks.rados.rados.0.smithi017.stdout:1073: oid 22 version 79 is already newer than 79 2023-03-29T14:06:47.356 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 79 (ObjNum 529 snap 120 seq_num 529) dirty exists 2023-03-29T14:06:47.356 INFO:tasks.rados.rados.0.smithi017.stdout:1073: left oid 22 (ObjNum 529 snap 120 seq_num 529) 2023-03-29T14:06:47.429 INFO:tasks.rados.rados.0.smithi017.stdout:1075: finishing write tid 1 to smithi017107884-6 2023-03-29T14:06:47.440 INFO:tasks.rados.rados.0.smithi017.stdout:1076: finishing write tid 1 to smithi017107884-32 2023-03-29T14:06:47.464 INFO:tasks.rados.rados.0.smithi017.stdout:1076: finishing write tid 2 to smithi017107884-32 2023-03-29T14:06:47.474 INFO:tasks.rados.rados.0.smithi017.stdout:1075: finishing write tid 2 to smithi017107884-6 2023-03-29T14:06:47.474 INFO:tasks.rados.rados.0.smithi017.stdout:1075: finishing write tid 3 to smithi017107884-6 2023-03-29T14:06:47.475 INFO:tasks.rados.rados.0.smithi017.stdout:1075: finishing write tid 4 to smithi017107884-6 2023-03-29T14:06:47.476 INFO:tasks.rados.rados.0.smithi017.stdout:1072: done (6 left) 2023-03-29T14:06:47.476 INFO:tasks.rados.rados.0.smithi017.stdout:1073: done (5 left) 2023-03-29T14:06:47.476 INFO:tasks.rados.rados.0.smithi017.stdout:1078: done (4 left) 2023-03-29T14:06:47.476 INFO:tasks.rados.rados.0.smithi017.stdout:1079: done (3 left) 2023-03-29T14:06:47.476 INFO:tasks.rados.rados.0.smithi017.stdout:1080: snap_create 2023-03-29T14:06:47.477 INFO:tasks.rados.rados.0.smithi017.stdout:1075: finishing write tid 5 to smithi017107884-6 2023-03-29T14:06:47.477 INFO:tasks.rados.rados.0.smithi017.stdout:1075: oid 6 updating version 0 to 115 2023-03-29T14:06:47.477 INFO:tasks.rados.rados.0.smithi017.stdout:1075: oid 6 version 115 is already newer than 115 2023-03-29T14:06:47.477 INFO:tasks.rados.rados.0.smithi017.stdout:1075: oid 6 version 115 is already newer than 115 2023-03-29T14:06:47.477 INFO:tasks.rados.rados.0.smithi017.stdout:1075: oid 6 version 115 is already newer than 115 2023-03-29T14:06:47.477 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 115 (ObjNum 530 snap 120 seq_num 530) dirty exists 2023-03-29T14:06:47.477 INFO:tasks.rados.rados.0.smithi017.stdout:1075: left oid 6 (ObjNum 530 snap 120 seq_num 530) 2023-03-29T14:06:47.490 INFO:tasks.rados.rados.0.smithi017.stdout:1077: expect (ObjNum 458 snap 103 seq_num 458) 2023-03-29T14:06:47.633 INFO:tasks.rados.rados.0.smithi017.stdout:1076: finishing write tid 3 to smithi017107884-32 2023-03-29T14:06:47.633 INFO:tasks.rados.rados.0.smithi017.stdout:1076: finishing write tid 4 to smithi017107884-32 2023-03-29T14:06:47.634 INFO:tasks.rados.rados.0.smithi017.stdout:1076: oid 32 updating version 0 to 217 2023-03-29T14:06:47.634 INFO:tasks.rados.rados.0.smithi017.stdout:1076: oid 32 version 217 is already newer than 215 2023-03-29T14:06:47.634 INFO:tasks.rados.rados.0.smithi017.stdout:1076: oid 32 version 217 is already newer than 217 2023-03-29T14:06:47.634 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 217 (ObjNum 531 snap 120 seq_num 531) dirty exists 2023-03-29T14:06:47.634 INFO:tasks.rados.rados.0.smithi017.stdout:1076: left oid 32 (ObjNum 531 snap 120 seq_num 531) 2023-03-29T14:06:48.226 INFO:tasks.rados.rados.0.smithi017.stdout:1075: done (3 left) 2023-03-29T14:06:48.226 INFO:tasks.rados.rados.0.smithi017.stdout:1076: done (2 left) 2023-03-29T14:06:48.226 INFO:tasks.rados.rados.0.smithi017.stdout:1077: done (1 left) 2023-03-29T14:06:48.226 INFO:tasks.rados.rados.0.smithi017.stdout:1080: done (0 left) 2023-03-29T14:06:48.226 INFO:tasks.rados.rados.0.smithi017.stdout:1081: delete oid 17 current snap is 121 2023-03-29T14:06:48.249 INFO:tasks.rados.rados.0.smithi017.stdout:1081: done (0 left) 2023-03-29T14:06:48.250 INFO:tasks.rados.rados.0.smithi017.stdout:1082: write oid 1 current snap is 121 2023-03-29T14:06:48.250 INFO:tasks.rados.rados.0.smithi017.stdout:1082: seq_num 534 ranges {574938=727676,1731855=703504,2848231=126707} 2023-03-29T14:06:48.256 INFO:tasks.rados.rados.0.smithi017.stdout:1082: writing smithi017107884-1 from 574938 to 1302614 tid 1 2023-03-29T14:06:48.263 INFO:tasks.rados.rados.0.smithi017.stdout:1082: writing smithi017107884-1 from 1731855 to 2435359 tid 2 2023-03-29T14:06:48.266 INFO:tasks.rados.rados.0.smithi017.stdout:1082: writing smithi017107884-1 from 2848231 to 2974938 tid 3 2023-03-29T14:06:48.266 INFO:tasks.rados.rados.0.smithi017.stdout:1083: snap_remove snap 120 2023-03-29T14:06:49.139 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~99s 2023-03-29T14:06:49.231 INFO:tasks.rados.rados.0.smithi017.stdout:1083: done (1 left) 2023-03-29T14:06:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:1084: read oid 7 snap 112 2023-03-29T14:06:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:1084: expect deleted 2023-03-29T14:06:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:1085: write (excl) oid 9 current snap is 121 2023-03-29T14:06:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:1082: finishing write tid 1 to smithi017107884-1 2023-03-29T14:06:49.232 INFO:tasks.rados.rados.0.smithi017.stdout:1085: seq_num 535 ranges {591745=402925,1790337=563515} 2023-03-29T14:06:49.236 INFO:tasks.rados.rados.0.smithi017.stdout:1085: writing smithi017107884-9 from 591745 to 994670 tid 1 2023-03-29T14:06:49.244 INFO:tasks.rados.rados.0.smithi017.stdout:1085: writing smithi017107884-9 from 1790337 to 2353852 tid 2 2023-03-29T14:06:49.244 INFO:tasks.rados.rados.0.smithi017.stdout:1086: read oid 47 snap -1 2023-03-29T14:06:49.244 INFO:tasks.rados.rados.0.smithi017.stdout:1086: expect deleted 2023-03-29T14:06:49.244 INFO:tasks.rados.rados.0.smithi017.stdout:1087: read oid 13 snap -1 2023-03-29T14:06:49.244 INFO:tasks.rados.rados.0.smithi017.stdout:1087: expect deleted 2023-03-29T14:06:49.244 INFO:tasks.rados.rados.0.smithi017.stdout:1088: read oid 20 snap -1 2023-03-29T14:06:49.245 INFO:tasks.rados.rados.0.smithi017.stdout:1088: expect (ObjNum 485 snap 111 seq_num 485) 2023-03-29T14:06:49.245 INFO:tasks.rados.rados.0.smithi017.stdout:1089: read oid 14 snap -1 2023-03-29T14:06:49.245 INFO:tasks.rados.rados.0.smithi017.stdout:1089: expect deleted 2023-03-29T14:06:49.245 INFO:tasks.rados.rados.0.smithi017.stdout:1090: write oid 8 current snap is 121 2023-03-29T14:06:49.245 INFO:tasks.rados.rados.0.smithi017.stdout:1090: seq_num 536 ranges {608552=478174,1848819=707173} 2023-03-29T14:06:49.250 INFO:tasks.rados.rados.0.smithi017.stdout:1090: writing smithi017107884-8 from 608552 to 1086726 tid 1 2023-03-29T14:06:49.258 INFO:tasks.rados.rados.0.smithi017.stdout:1090: writing smithi017107884-8 from 1848819 to 2555992 tid 2 2023-03-29T14:06:49.258 INFO:tasks.rados.rados.0.smithi017.stdout:1091: read oid 32 snap -1 2023-03-29T14:06:49.258 INFO:tasks.rados.rados.0.smithi017.stdout:1091: expect (ObjNum 531 snap 120 seq_num 531) 2023-03-29T14:06:49.258 INFO:tasks.rados.rados.0.smithi017.stdout:1092: read oid 17 snap -1 2023-03-29T14:06:49.259 INFO:tasks.rados.rados.0.smithi017.stdout:1092: expect deleted 2023-03-29T14:06:49.259 INFO:tasks.rados.rados.0.smithi017.stdout:1093: snap_remove snap 117 2023-03-29T14:06:49.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:49.421+0000 7fd6247e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:49.427+0000 7fd6247e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:49.511 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:49.512 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:50.232 INFO:tasks.rados.rados.0.smithi017.stdout:1093: done (10 left) 2023-03-29T14:06:50.232 INFO:tasks.rados.rados.0.smithi017.stdout:1094: read oid 27 snap -1 2023-03-29T14:06:50.232 INFO:tasks.rados.rados.0.smithi017.stdout:1094: expect (ObjNum 458 snap 103 seq_num 458) 2023-03-29T14:06:50.232 INFO:tasks.rados.rados.0.smithi017.stdout:1095: read oid 40 snap -1 2023-03-29T14:06:50.232 INFO:tasks.rados.rados.0.smithi017.stdout:1095: expect deleted 2023-03-29T14:06:50.232 INFO:tasks.rados.rados.0.smithi017.stdout:1096: write oid 22 current snap is 121 2023-03-29T14:06:50.233 INFO:tasks.rados.rados.0.smithi017.stdout:1082: finishing write tid 2 to smithi017107884-1 2023-03-29T14:06:50.233 INFO:tasks.rados.rados.0.smithi017.stdout:1096: seq_num 537 ranges {625359=553423,1790948=567184,2930500=94859} 2023-03-29T14:06:50.238 INFO:tasks.rados.rados.0.smithi017.stdout:1096: writing smithi017107884-22 from 625359 to 1178782 tid 1 2023-03-29T14:06:50.245 INFO:tasks.rados.rados.0.smithi017.stdout:1096: writing smithi017107884-22 from 1790948 to 2358132 tid 2 2023-03-29T14:06:50.248 INFO:tasks.rados.rados.0.smithi017.stdout:1096: writing smithi017107884-22 from 2930500 to 3025359 tid 3 2023-03-29T14:06:50.249 INFO:tasks.rados.rados.0.smithi017.stdout:1097: write oid 34 current snap is 121 2023-03-29T14:06:50.249 INFO:tasks.rados.rados.0.smithi017.stdout:1097: seq_num 538 ranges {642166=628672,1849430=710842} 2023-03-29T14:06:50.255 INFO:tasks.rados.rados.0.smithi017.stdout:1097: writing smithi017107884-34 from 642166 to 1270838 tid 1 2023-03-29T14:06:50.263 INFO:tasks.rados.rados.0.smithi017.stdout:1097: writing smithi017107884-34 from 1849430 to 2560272 tid 2 2023-03-29T14:06:50.263 INFO:tasks.rados.rados.0.smithi017.stdout:1098: write (excl) oid 10 current snap is 121 2023-03-29T14:06:50.263 INFO:tasks.rados.rados.0.smithi017.stdout:1098: seq_num 539 ranges {658973=703921,1907912=570853,2985346=73627} 2023-03-29T14:06:50.270 INFO:tasks.rados.rados.0.smithi017.stdout:1098: writing smithi017107884-10 from 658973 to 1362894 tid 1 2023-03-29T14:06:50.276 INFO:tasks.rados.rados.0.smithi017.stdout:1098: writing smithi017107884-10 from 1907912 to 2478765 tid 2 2023-03-29T14:06:50.280 INFO:tasks.rados.rados.0.smithi017.stdout:1098: writing smithi017107884-10 from 2985346 to 3058973 tid 3 2023-03-29T14:06:50.280 INFO:tasks.rados.rados.0.smithi017.stdout:1099: snap_create 2023-03-29T14:06:50.280 INFO:tasks.rados.rados.0.smithi017.stdout:1082: finishing write tid 3 to smithi017107884-1 2023-03-29T14:06:50.280 INFO:tasks.rados.rados.0.smithi017.stdout:1082: finishing write tid 4 to smithi017107884-1 2023-03-29T14:06:50.280 INFO:tasks.rados.rados.0.smithi017.stdout:1082: finishing write tid 5 to smithi017107884-1 2023-03-29T14:06:50.281 INFO:tasks.rados.rados.0.smithi017.stdout:1082: oid 1 updating version 0 to 150 2023-03-29T14:06:50.281 INFO:tasks.rados.rados.0.smithi017.stdout:1082: oid 1 version 150 is already newer than 150 2023-03-29T14:06:50.281 INFO:tasks.rados.rados.0.smithi017.stdout:1082: oid 1 version 150 is already newer than 150 2023-03-29T14:06:50.281 INFO:tasks.rados.rados.0.smithi017.stdout:1082: oid 1 version 150 is already newer than 150 2023-03-29T14:06:50.281 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 150 (ObjNum 534 snap 121 seq_num 534) dirty exists 2023-03-29T14:06:50.281 INFO:tasks.rados.rados.0.smithi017.stdout:1082: left oid 1 (ObjNum 534 snap 121 seq_num 534) 2023-03-29T14:06:50.281 INFO:tasks.rados.rados.0.smithi017.stdout:1085: finishing write tid 1 to smithi017107884-9 2023-03-29T14:06:50.282 INFO:tasks.rados.rados.0.smithi017.stdout:1091: expect (ObjNum 531 snap 120 seq_num 531) 2023-03-29T14:06:50.457 INFO:tasks.rados.rados.0.smithi017.stdout:1090: finishing write tid 1 to smithi017107884-8 2023-03-29T14:06:50.457 INFO:tasks.rados.rados.0.smithi017.stdout:1090: finishing write tid 2 to smithi017107884-8 2023-03-29T14:06:50.457 INFO:tasks.rados.rados.0.smithi017.stdout:1085: finishing write tid 2 to smithi017107884-9 2023-03-29T14:06:50.457 INFO:tasks.rados.rados.0.smithi017.stdout:1085: finishing write tid 3 to smithi017107884-9 2023-03-29T14:06:50.457 INFO:tasks.rados.rados.0.smithi017.stdout:1085: finishing write tid 4 to smithi017107884-9 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1085: oid 9 updating version 0 to 154 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1085: oid 9 version 154 is already newer than 154 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1085: oid 9 version 154 is already newer than 152 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 154 (ObjNum 535 snap 121 seq_num 535) dirty exists 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1085: left oid 9 (ObjNum 535 snap 121 seq_num 535) 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1090: finishing write tid 3 to smithi017107884-8 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1090: finishing write tid 4 to smithi017107884-8 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1090: oid 8 updating version 0 to 154 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1090: oid 8 version 154 is already newer than 154 2023-03-29T14:06:50.458 INFO:tasks.rados.rados.0.smithi017.stdout:1090: oid 8 version 154 is already newer than 154 2023-03-29T14:06:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 154 (ObjNum 536 snap 121 seq_num 536) dirty exists 2023-03-29T14:06:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:1090: left oid 8 (ObjNum 536 snap 121 seq_num 536) 2023-03-29T14:06:50.459 INFO:tasks.rados.rados.0.smithi017.stdout:1088: expect (ObjNum 485 snap 111 seq_num 485) 2023-03-29T14:06:50.647 INFO:tasks.rados.rados.0.smithi017.stdout:1094: expect (ObjNum 458 snap 103 seq_num 458) 2023-03-29T14:06:50.902 INFO:tasks.rados.rados.0.smithi017.stdout:1096: finishing write tid 1 to smithi017107884-22 2023-03-29T14:06:50.902 INFO:tasks.rados.rados.0.smithi017.stdout:1096: finishing write tid 2 to smithi017107884-22 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: finishing write tid 3 to smithi017107884-22 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: finishing write tid 4 to smithi017107884-22 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: finishing write tid 5 to smithi017107884-22 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: oid 22 updating version 0 to 83 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: oid 22 version 83 is already newer than 83 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: oid 22 version 83 is already newer than 82 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: oid 22 version 83 is already newer than 83 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 83 (ObjNum 537 snap 121 seq_num 537) dirty exists 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1096: left oid 22 (ObjNum 537 snap 121 seq_num 537) 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1098: finishing write tid 1 to smithi017107884-10 2023-03-29T14:06:50.903 INFO:tasks.rados.rados.0.smithi017.stdout:1097: finishing write tid 1 to smithi017107884-34 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: finishing write tid 2 to smithi017107884-10 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: finishing write tid 3 to smithi017107884-10 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1097: finishing write tid 2 to smithi017107884-34 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: finishing write tid 4 to smithi017107884-10 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: finishing write tid 5 to smithi017107884-10 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: oid 10 updating version 0 to 154 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: oid 10 version 154 is already newer than 151 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: oid 10 version 154 is already newer than 154 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: oid 10 version 154 is already newer than 154 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 154 (ObjNum 539 snap 121 seq_num 539) dirty exists 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1098: left oid 10 (ObjNum 539 snap 121 seq_num 539) 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1097: finishing write tid 3 to smithi017107884-34 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1097: finishing write tid 4 to smithi017107884-34 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1097: oid 34 updating version 0 to 165 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1097: oid 34 version 165 is already newer than 164 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1097: oid 34 version 165 is already newer than 165 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 165 (ObjNum 538 snap 121 seq_num 538) dirty exists 2023-03-29T14:06:50.904 INFO:tasks.rados.rados.0.smithi017.stdout:1097: left oid 34 (ObjNum 538 snap 121 seq_num 538) 2023-03-29T14:06:51.231 INFO:tasks.rados.rados.0.smithi017.stdout:1082: done (15 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1084: done (14 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1085: done (13 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1086: done (12 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1087: done (11 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1088: done (10 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1089: done (9 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1090: done (8 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1091: done (7 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1092: done (6 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1094: done (5 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1095: done (4 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1096: done (3 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1097: done (2 left) 2023-03-29T14:06:51.232 INFO:tasks.rados.rados.0.smithi017.stdout:1098: done (1 left) 2023-03-29T14:06:51.233 INFO:tasks.rados.rados.0.smithi017.stdout:1099: done (0 left) 2023-03-29T14:06:51.233 INFO:tasks.rados.rados.0.smithi017.stdout:1100: write oid 2 current snap is 122 2023-03-29T14:06:51.233 INFO:tasks.rados.rados.0.smithi017.stdout:1100: seq_num 540 ranges {675780=495523,1682747=714511,3012769=63011} 2023-03-29T14:06:51.241 INFO:tasks.rados.rados.0.smithi017.stdout:1100: writing smithi017107884-2 from 675780 to 1171303 tid 1 2023-03-29T14:06:51.244 INFO:tasks.rados.rados.0.smithi017.stdout:1100: writing smithi017107884-2 from 1682747 to 2397258 tid 2 2023-03-29T14:06:51.248 INFO:tasks.rados.rados.0.smithi017.stdout:1100: writing smithi017107884-2 from 3012769 to 3075780 tid 3 2023-03-29T14:06:51.248 INFO:tasks.rados.rados.0.smithi017.stdout:1101: read oid 6 snap -1 2023-03-29T14:06:51.248 INFO:tasks.rados.rados.0.smithi017.stdout:1101: expect (ObjNum 530 snap 120 seq_num 530) 2023-03-29T14:06:51.248 INFO:tasks.rados.rados.0.smithi017.stdout:1102: delete oid 13 current snap is 122 2023-03-29T14:06:51.289 INFO:tasks.rados.rados.0.smithi017.stdout:1101: expect (ObjNum 530 snap 120 seq_num 530) 2023-03-29T14:06:51.390 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:51.390 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:51.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:51.477 INFO:tasks.rados.rados.0.smithi017.stdout:1100: finishing write tid 1 to smithi017107884-2 2023-03-29T14:06:51.477 INFO:tasks.rados.rados.0.smithi017.stdout:1100: finishing write tid 2 to smithi017107884-2 2023-03-29T14:06:51.477 INFO:tasks.rados.rados.0.smithi017.stdout:1101: done (2 left) 2023-03-29T14:06:51.477 INFO:tasks.rados.rados.0.smithi017.stdout:1102: done (1 left) 2023-03-29T14:06:51.477 INFO:tasks.rados.rados.0.smithi017.stdout:1103: snap_remove snap 121 2023-03-29T14:06:51.513 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:51.521 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:51.518+0000 7f694f0a2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:51.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:51.525+0000 7f694f0a2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:51.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:51.643+0000 7f264593b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:51.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:51.647+0000 7f264593b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:51.709 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:51.712 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:51.712 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:51.732 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:51.746 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2023-03-29T14:06:51.230023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134599256,"num_objects":586,"num_object_clones":490,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":48,"num_read":2490,"num_read_kb":1398966,"num_write":3348,"num_write_kb":897096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1762,"ondisk_log_size":1762,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1148844,"kb_used_data":1090060,"kb_used_omap":0,"kb_used_meta":58752,"kb_avail":376338516,"statfs":{"total":386547056640,"available":385370640384,"internally_reserved":0,"allocated":1116221440,"data_stored":1340871254,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":60162048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.908922"},"pg_stats":[{"pgid":"3.f","version":"260'75","reported_seq":162,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:46.439622+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:46.439622+0000","last_peered":"2023-03-29T14:06:46.439622+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:46.439622+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12235826,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":28057,"num_write":70,"num_write_kb":20314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":8}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"256'32","reported_seq":274,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:42.495993+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:42.495993+0000","last_peered":"2023-03-29T14:06:42.495993+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:42.495993+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5580122,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":25430,"num_write":62,"num_write_kb":18414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"252'149","reported_seq":546,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-03-29T14:06:45.346311+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:45.346311+0000","last_peered":"2023-03-29T14:06:45.346311+0000","last_clean":"2023-03-29T14:06:45.346311+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:45.346311+0000","last_undegraded":"2023-03-29T14:06:45.346311+0000","last_fullsized":"2023-03-29T14:06:45.346311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748456,"num_objects":52,"num_object_clones":42,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":218,"num_read_kb":143078,"num_write":280,"num_write_kb":82206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'151","reported_seq":555,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-03-29T14:06:45.353444+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:45.353444+0000","last_peered":"2023-03-29T14:06:45.353444+0000","last_clean":"2023-03-29T14:06:45.353444+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:45.353444+0000","last_undegraded":"2023-03-29T14:06:45.353444+0000","last_fullsized":"2023-03-29T14:06:45.353444+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5042298,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":230,"num_read_kb":141088,"num_write":292,"num_write_kb":82024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"255'161","reported_seq":553,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-03-29T14:06:42.341463+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:42.341463+0000","last_peered":"2023-03-29T14:06:42.341463+0000","last_clean":"2023-03-29T14:06:42.341463+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:42.341463+0000","last_undegraded":"2023-03-29T14:06:42.341463+0000","last_fullsized":"2023-03-29T14:06:42.341463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14252930,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":201,"num_read_kb":110014,"num_write":326,"num_write_kb":94240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":217,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-03-29T14:06:45.339530+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:45.339530+0000","last_peered":"2023-03-29T14:06:45.339530+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:45.339530+0000","last_undegraded":"2023-03-29T14:06:45.339530+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"254'100","reported_seq":430,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:45.344459+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:45.344459+0000","last_peered":"2023-03-29T14:06:45.344459+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:45.344459+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":126,"num_read_kb":61238,"num_write":180,"num_write_kb":52872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'69","reported_seq":364,"reported_epoch":260,"state":"active+clean","last_fresh":"2023-03-29T14:06:45.349483+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:45.349483+0000","last_peered":"2023-03-29T14:06:45.349483+0000","last_clean":"2023-03-29T14:06:45.349483+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:45.349483+0000","last_undegraded":"2023-03-29T14:06:45.349483+0000","last_fullsized":"2023-03-29T14:06:45.349483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":88,"num_read_kb":49143,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.352572+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:42.352572+0000","last_peered":"2023-03-29T14:06:42.352572+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:42.352572+0000","last_undegraded":"2023-03-29T14:06:42.352572+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"259'58","reported_seq":113,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:45.454673+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:06:45.454673+0000","last_peered":"2023-03-29T14:06:45.454673+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:45.454673+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706290,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":12587,"num_write":42,"num_write_kb":11134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":220,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-03-29T14:06:45.351414+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:45.351414+0000","last_peered":"2023-03-29T14:06:45.351414+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:45.351414+0000","last_undegraded":"2023-03-29T14:06:45.351414+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":390,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:44.387638+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:44.387638+0000","last_peered":"2023-03-29T14:06:44.387638+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:44.387638+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.364703+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:42.364703+0000","last_peered":"2023-03-29T14:06:42.364703+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:42.364703+0000","last_undegraded":"2023-03-29T14:06:42.364703+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"237'91","reported_seq":401,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:42.374414+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:42.374414+0000","last_peered":"2023-03-29T14:06:42.374414+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:42.374414+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":110,"num_read_kb":50873,"num_write":178,"num_write_kb":53770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":220,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-03-29T14:06:45.346890+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:45.346890+0000","last_peered":"2023-03-29T14:06:45.346890+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:45.346890+0000","last_undegraded":"2023-03-29T14:06:45.346890+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":431,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:42.354547+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:42.354547+0000","last_peered":"2023-03-29T14:06:42.354547+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:42.354547+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"254'111","reported_seq":509,"reported_epoch":260,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:45.340732+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:45.340732+0000","last_peered":"2023-03-29T14:06:45.340732+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:45.340732+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17479808,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":236,"num_read_kb":156411,"num_write":222,"num_write_kb":64270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-03-29T14:06:45.338229+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:45.338229+0000","last_peered":"2023-03-29T14:06:45.338229+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:45.338229+0000","last_undegraded":"2023-03-29T14:06:45.338229+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"251'209","reported_seq":699,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:44.390637+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:44.390637+0000","last_peered":"2023-03-29T14:06:44.390637+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:44.390637+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975796,"num_objects":72,"num_object_clones":58,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":365,"num_read_kb":227364,"num_write":388,"num_write_kb":111828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":72}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":206,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.356526+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:42.356526+0000","last_peered":"2023-03-29T14:06:42.356526+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:42.356526+0000","last_undegraded":"2023-03-29T14:06:42.356526+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"249'75","reported_seq":376,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-03-29T14:06:42.360696+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:42.360696+0000","last_peered":"2023-03-29T14:06:42.360696+0000","last_clean":"2023-03-29T14:06:42.360696+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:42.360696+0000","last_undegraded":"2023-03-29T14:06:42.360696+0000","last_fullsized":"2023-03-29T14:06:42.360696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4470794,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":70176,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"254'160","reported_seq":549,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-03-29T14:06:42.336613+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:42.336613+0000","last_peered":"2023-03-29T14:06:42.336613+0000","last_clean":"2023-03-29T14:06:42.336613+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:42.336613+0000","last_undegraded":"2023-03-29T14:06:42.336613+0000","last_fullsized":"2023-03-29T14:06:42.336613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958758,"num_objects":56,"num_object_clones":46,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":200,"num_read_kb":115324,"num_write":306,"num_write_kb":89642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":206,"reported_epoch":259,"state":"active+undersized","last_fresh":"2023-03-29T14:06:42.362324+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:42.362324+0000","last_peered":"2023-03-29T14:06:42.362324+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:42.362324+0000","last_undegraded":"2023-03-29T14:06:42.362324+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":260,"state":"active+undersized","last_fresh":"2023-03-29T14:06:45.348079+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:45.348079+0000","last_peered":"2023-03-29T14:06:45.348079+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:45.348079+0000","last_undegraded":"2023-03-29T14:06:45.348079+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"249'145","reported_seq":530,"reported_epoch":259,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:44.371658+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:44.371658+0000","last_peered":"2023-03-29T14:06:44.371658+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:44.371658+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12034142,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":201,"num_read_kb":119527,"num_write":282,"num_write_kb":85334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":133418520,"num_objects":582,"num_object_clones":490,"num_object_copies":1746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":48,"num_read":2348,"num_read_kb":1398846,"num_write":3074,"num_write_kb":893308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1665,"ondisk_log_size":1665,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312024,"kb_used_data":297104,"kb_used_omap":0,"kb_used_meta":14912,"kb_avail":94059816,"statfs":{"total":96636764160,"available":96317251584,"internally_reserved":0,"allocated":304234496,"data_stored":362510272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15269888},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":354636,"kb_used_data":338564,"kb_used_omap":0,"kb_used_meta":16064,"kb_avail":94017204,"statfs":{"total":96636764160,"available":96273616896,"internally_reserved":0,"allocated":346689536,"data_stored":419265414,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16449536},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":326240,"kb_used_data":310552,"kb_used_omap":0,"kb_used_meta":15680,"kb_avail":94045600,"statfs":{"total":96636764160,"available":96302694400,"internally_reserved":0,"allocated":318005248,"data_stored":385863343,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16056320},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:51.751 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.5 1.0 3.1 3.6 2023-03-29T14:06:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:51.914+0000 7f96f0100700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:51.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:51.924+0000 7f96f0100700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:52.135 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.5,3.6] on osd.0 to force-recovery; instructing pg(s) [1.0] on osd.1 to force-recovery; 2023-03-29T14:06:52.135 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:06:52.342 INFO:tasks.rados.rados.0.smithi017.stdout:1103: done (1 left) 2023-03-29T14:06:52.342 INFO:tasks.rados.rados.0.smithi017.stdout:1104: snap_remove snap 115 2023-03-29T14:06:53.344 INFO:tasks.rados.rados.0.smithi017.stdout:1104: done (1 left) 2023-03-29T14:06:53.344 INFO:tasks.rados.rados.0.smithi017.stdout:1105: snap_create 2023-03-29T14:06:53.344 INFO:tasks.rados.rados.0.smithi017.stdout:1100: finishing write tid 3 to smithi017107884-2 2023-03-29T14:06:53.344 INFO:tasks.rados.rados.0.smithi017.stdout:1100: finishing write tid 4 to smithi017107884-2 2023-03-29T14:06:53.344 INFO:tasks.rados.rados.0.smithi017.stdout:1100: finishing write tid 5 to smithi017107884-2 2023-03-29T14:06:53.344 INFO:tasks.rados.rados.0.smithi017.stdout:1100: oid 2 updating version 0 to 221 2023-03-29T14:06:53.344 INFO:tasks.rados.rados.0.smithi017.stdout:1100: oid 2 version 221 is already newer than 220 2023-03-29T14:06:53.345 INFO:tasks.rados.rados.0.smithi017.stdout:1100: oid 2 updating version 221 to 222 2023-03-29T14:06:53.345 INFO:tasks.rados.rados.0.smithi017.stdout:1100: oid 2 version 222 is already newer than 222 2023-03-29T14:06:53.345 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 222 (ObjNum 540 snap 122 seq_num 540) dirty exists 2023-03-29T14:06:53.345 INFO:tasks.rados.rados.0.smithi017.stdout:1100: left oid 2 (ObjNum 540 snap 122 seq_num 540) 2023-03-29T14:06:53.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:53.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:53.842+0000 7f2d5c922700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:53.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:53.846+0000 7f2d5c922700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:53.909 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:53.911 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:53.911 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:54.353 INFO:tasks.rados.rados.0.smithi017.stdout:1100: done (1 left) 2023-03-29T14:06:54.353 INFO:tasks.rados.rados.0.smithi017.stdout:1105: done (0 left) 2023-03-29T14:06:54.353 INFO:tasks.rados.rados.0.smithi017.stdout:1106: write (excl) oid 40 current snap is 123 2023-03-29T14:06:54.353 INFO:tasks.rados.rados.0.smithi017.stdout:1106: seq_num 542 ranges {709394=646021,2083358=718180} 2023-03-29T14:06:54.360 INFO:tasks.rados.rados.0.smithi017.stdout:1106: writing smithi017107884-40 from 709394 to 1355415 tid 1 2023-03-29T14:06:54.368 INFO:tasks.rados.rados.0.smithi017.stdout:1106: writing smithi017107884-40 from 2083358 to 2801538 tid 2 2023-03-29T14:06:54.369 INFO:tasks.rados.rados.0.smithi017.stdout:1107: write (excl) oid 48 current snap is 123 2023-03-29T14:06:54.369 INFO:tasks.rados.rados.0.smithi017.stdout:1107: seq_num 543 ranges {726201=721270,2141840=578191} 2023-03-29T14:06:54.376 INFO:tasks.rados.rados.0.smithi017.stdout:1107: writing smithi017107884-48 from 726201 to 1447471 tid 1 2023-03-29T14:06:54.383 INFO:tasks.rados.rados.0.smithi017.stdout:1107: writing smithi017107884-48 from 2141840 to 2720031 tid 2 2023-03-29T14:06:54.383 INFO:tasks.rados.rados.0.smithi017.stdout:1108: write (excl) oid 13 current snap is 123 2023-03-29T14:06:54.383 INFO:tasks.rados.rados.0.smithi017.stdout:1108: seq_num 544 ranges {743008=796519,2200322=721849} 2023-03-29T14:06:54.392 INFO:tasks.rados.rados.0.smithi017.stdout:1108: writing smithi017107884-13 from 743008 to 1539527 tid 1 2023-03-29T14:06:54.400 INFO:tasks.rados.rados.0.smithi017.stdout:1108: writing smithi017107884-13 from 2200322 to 2922171 tid 2 2023-03-29T14:06:54.400 INFO:tasks.rados.rados.0.smithi017.stdout:1109: write (excl) oid 43 current snap is 123 2023-03-29T14:06:54.400 INFO:tasks.rados.rados.0.smithi017.stdout:1109: seq_num 545 ranges {759815=471768,1742451=465507,2917178=242637} 2023-03-29T14:06:54.406 INFO:tasks.rados.rados.0.smithi017.stdout:1109: writing smithi017107884-43 from 759815 to 1231583 tid 1 2023-03-29T14:06:54.412 INFO:tasks.rados.rados.0.smithi017.stdout:1109: writing smithi017107884-43 from 1742451 to 2207958 tid 2 2023-03-29T14:06:54.417 INFO:tasks.rados.rados.0.smithi017.stdout:1109: writing smithi017107884-43 from 2917178 to 3159815 tid 3 2023-03-29T14:06:54.417 INFO:tasks.rados.rados.0.smithi017.stdout:1110: snap_remove snap 119 2023-03-29T14:06:54.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~105s 2023-03-29T14:06:55.358 INFO:tasks.rados.rados.0.smithi017.stdout:1110: done (4 left) 2023-03-29T14:06:55.358 INFO:tasks.rados.rados.0.smithi017.stdout:1111: write oid 3 current snap is 123 2023-03-29T14:06:55.358 INFO:tasks.rados.rados.0.smithi017.stdout:1106: finishing write tid 1 to smithi017107884-40 2023-03-29T14:06:55.358 INFO:tasks.rados.rados.0.smithi017.stdout:1111: seq_num 546 ranges {776622=547017,1800933=725518,2944601=232021} 2023-03-29T14:06:55.365 INFO:tasks.rados.rados.0.smithi017.stdout:1111: writing smithi017107884-3 from 776622 to 1323639 tid 1 2023-03-29T14:06:55.372 INFO:tasks.rados.rados.0.smithi017.stdout:1111: writing smithi017107884-3 from 1800933 to 2526451 tid 2 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1111: writing smithi017107884-3 from 2944601 to 3176622 tid 3 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1112: snap_create 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1107: finishing write tid 1 to smithi017107884-48 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1106: finishing write tid 2 to smithi017107884-40 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1106: finishing write tid 3 to smithi017107884-40 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1106: finishing write tid 4 to smithi017107884-40 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1106: oid 40 updating version 0 to 157 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1106: oid 40 version 157 is already newer than 157 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1106: oid 40 version 157 is already newer than 155 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 157 (ObjNum 542 snap 123 seq_num 542) dirty exists 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1106: left oid 40 (ObjNum 542 snap 123 seq_num 542) 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1108: finishing write tid 1 to smithi017107884-13 2023-03-29T14:06:55.376 INFO:tasks.rados.rados.0.smithi017.stdout:1109: finishing write tid 1 to smithi017107884-43 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1107: finishing write tid 2 to smithi017107884-48 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1108: finishing write tid 2 to smithi017107884-13 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: finishing write tid 2 to smithi017107884-43 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: finishing write tid 3 to smithi017107884-43 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1108: finishing write tid 3 to smithi017107884-13 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1108: finishing write tid 4 to smithi017107884-13 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1108: oid 13 updating version 0 to 167 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1108: oid 13 version 167 is already newer than 167 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1108: oid 13 version 167 is already newer than 165 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 167 (ObjNum 544 snap 123 seq_num 544) dirty exists 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1108: left oid 13 (ObjNum 544 snap 123 seq_num 544) 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: finishing write tid 4 to smithi017107884-43 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: finishing write tid 5 to smithi017107884-43 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: oid 43 updating version 0 to 96 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: oid 43 version 96 is already newer than 96 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: oid 43 version 96 is already newer than 96 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:1109: oid 43 version 96 is already newer than 93 2023-03-29T14:06:55.377 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 96 (ObjNum 545 snap 123 seq_num 545) dirty exists 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:1109: left oid 43 (ObjNum 545 snap 123 seq_num 545) 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:1107: finishing write tid 3 to smithi017107884-48 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:1107: finishing write tid 4 to smithi017107884-48 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:1107: oid 48 updating version 0 to 158 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:1107: oid 48 version 158 is already newer than 155 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:1107: oid 48 version 158 is already newer than 158 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 158 (ObjNum 543 snap 123 seq_num 543) dirty exists 2023-03-29T14:06:55.378 INFO:tasks.rados.rados.0.smithi017.stdout:1107: left oid 48 (ObjNum 543 snap 123 seq_num 543) 2023-03-29T14:06:55.498 INFO:tasks.rados.rados.0.smithi017.stdout:1111: finishing write tid 1 to smithi017107884-3 2023-03-29T14:06:55.499 INFO:tasks.rados.rados.0.smithi017.stdout:1111: finishing write tid 2 to smithi017107884-3 2023-03-29T14:06:55.501 INFO:tasks.rados.rados.0.smithi017.stdout:1111: finishing write tid 3 to smithi017107884-3 2023-03-29T14:06:55.503 INFO:tasks.rados.rados.0.smithi017.stdout:1111: finishing write tid 4 to smithi017107884-3 2023-03-29T14:06:55.505 INFO:tasks.rados.rados.0.smithi017.stdout:1111: finishing write tid 5 to smithi017107884-3 2023-03-29T14:06:55.505 INFO:tasks.rados.rados.0.smithi017.stdout:1111: oid 3 updating version 0 to 226 2023-03-29T14:06:55.505 INFO:tasks.rados.rados.0.smithi017.stdout:1111: oid 3 version 226 is already newer than 226 2023-03-29T14:06:55.505 INFO:tasks.rados.rados.0.smithi017.stdout:1111: oid 3 version 226 is already newer than 226 2023-03-29T14:06:55.505 INFO:tasks.rados.rados.0.smithi017.stdout:1111: oid 3 version 226 is already newer than 226 2023-03-29T14:06:55.505 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 226 (ObjNum 546 snap 123 seq_num 546) dirty exists 2023-03-29T14:06:55.505 INFO:tasks.rados.rados.0.smithi017.stdout:1111: left oid 3 (ObjNum 546 snap 123 seq_num 546) 2023-03-29T14:06:55.912 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:06:56.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:56.041+0000 7fbfc36e6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:56.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:56.045+0000 7fbfc36e6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:56.106 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:56.109 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:56.110 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:56.359 INFO:tasks.rados.rados.0.smithi017.stdout:1106: done (5 left) 2023-03-29T14:06:56.360 INFO:tasks.rados.rados.0.smithi017.stdout:1107: done (4 left) 2023-03-29T14:06:56.360 INFO:tasks.rados.rados.0.smithi017.stdout:1108: done (3 left) 2023-03-29T14:06:56.360 INFO:tasks.rados.rados.0.smithi017.stdout:1109: done (2 left) 2023-03-29T14:06:56.360 INFO:tasks.rados.rados.0.smithi017.stdout:1111: done (1 left) 2023-03-29T14:06:56.360 INFO:tasks.rados.rados.0.smithi017.stdout:1112: done (0 left) 2023-03-29T14:06:56.360 INFO:tasks.rados.rados.0.smithi017.stdout:1113: snap_create 2023-03-29T14:06:57.147 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:06:57.148 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:06:57.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:06:57.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:57.275+0000 7f494e1aa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:57.281 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:57.281+0000 7f494e1aa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:57.365 INFO:tasks.rados.rados.0.smithi017.stdout:1113: done (0 left) 2023-03-29T14:06:57.365 INFO:tasks.rados.rados.0.smithi017.stdout:1114: read oid 17 snap -1 2023-03-29T14:06:57.365 INFO:tasks.rados.rados.0.smithi017.stdout:1114: expect deleted 2023-03-29T14:06:57.365 INFO:tasks.rados.rados.0.smithi017.stdout:1115: write oid 12 current snap is 125 2023-03-29T14:06:57.365 INFO:tasks.rados.rados.0.smithi017.stdout:1115: seq_num 547 ranges {793429=622266,1859415=469176,2972024=221405} 2023-03-29T14:06:57.373 INFO:tasks.rados.rados.0.smithi017.stdout:1115: writing smithi017107884-12 from 793429 to 1415695 tid 1 2023-03-29T14:06:57.378 INFO:tasks.rados.rados.0.smithi017.stdout:1115: writing smithi017107884-12 from 1859415 to 2328591 tid 2 2023-03-29T14:06:57.382 INFO:tasks.rados.rados.0.smithi017.stdout:1115: writing smithi017107884-12 from 2972024 to 3193429 tid 3 2023-03-29T14:06:57.383 INFO:tasks.rados.rados.0.smithi017.stdout:1116: read oid 7 snap -1 2023-03-29T14:06:57.383 INFO:tasks.rados.rados.0.smithi017.stdout:1116: expect (ObjNum 521 snap 116 seq_num 521) 2023-03-29T14:06:57.383 INFO:tasks.rados.rados.0.smithi017.stdout:1117: snap_remove snap 123 2023-03-29T14:06:57.492 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:06:57.495 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:06:57.507 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2023-03-29T14:06:57.362332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":194837458,"num_objects":602,"num_object_clones":506,"num_object_copies":1806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":2568,"num_read_kb":1438214,"num_write":3512,"num_write_kb":941690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1840,"ondisk_log_size":1840,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1203088,"kb_used_data":1143536,"kb_used_omap":0,"kb_used_meta":59520,"kb_avail":376284272,"statfs":{"total":386547056640,"available":385315094528,"internally_reserved":0,"allocated":1170980864,"data_stored":1405376806,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":60948480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25782796,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-8,"num_read":14,"num_read_kb":6,"num_write":54,"num_write_kb":15908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020988"},"pg_stats":[{"pgid":"3.f","version":"264'83","reported_seq":186,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:54.375375+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:06:54.375375+0000","last_peered":"2023-03-29T14:06:54.375375+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:06:54.375375+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698068,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":28059,"num_write":88,"num_write_kb":25012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":10}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"256'32","reported_seq":281,"reported_epoch":268,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:53.244311+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:53.244311+0000","last_peered":"2023-03-29T14:06:53.244311+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:53.244311+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5580122,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":25430,"num_write":62,"num_write_kb":18414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"268'158","reported_seq":571,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-03-29T14:06:54.689524+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:06:54.689524+0000","last_peered":"2023-03-29T14:06:54.689524+0000","last_clean":"2023-03-29T14:06:54.689524+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:06:54.689524+0000","last_undegraded":"2023-03-29T14:06:54.689524+0000","last_fullsized":"2023-03-29T14:06:54.689524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16975664,"num_objects":54,"num_object_clones":44,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":224,"num_read_kb":143080,"num_write":298,"num_write_kb":87382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"268'157","reported_seq":577,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-03-29T14:06:54.594472+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:06:54.594472+0000","last_peered":"2023-03-29T14:06:54.594472+0000","last_clean":"2023-03-29T14:06:54.594472+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:06:54.594472+0000","last_undegraded":"2023-03-29T14:06:54.594472+0000","last_fullsized":"2023-03-29T14:06:54.594472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244576,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":236,"num_read_kb":141090,"num_write":308,"num_write_kb":86580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"264'165","reported_seq":570,"reported_epoch":268,"state":"active+clean","last_fresh":"2023-03-29T14:06:53.235413+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:53.235413+0000","last_peered":"2023-03-29T14:06:53.235413+0000","last_clean":"2023-03-29T14:06:53.235413+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:53.235413+0000","last_undegraded":"2023-03-29T14:06:53.235413+0000","last_fullsized":"2023-03-29T14:06:53.235413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15832788,"num_objects":62,"num_object_clones":54,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":209,"num_read_kb":115084,"num_write":334,"num_write_kb":96858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":225,"reported_epoch":269,"state":"active+undersized","last_fresh":"2023-03-29T14:06:54.373963+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:06:54.373963+0000","last_peered":"2023-03-29T14:06:54.373963+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:06:54.373963+0000","last_undegraded":"2023-03-29T14:06:54.373963+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"260'104","reported_seq":447,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:54.383054+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:06:54.383054+0000","last_peered":"2023-03-29T14:06:54.383054+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:06:54.383054+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748192,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":127,"num_read_kb":61239,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'69","reported_seq":372,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-03-29T14:06:54.380679+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:06:54.380679+0000","last_peered":"2023-03-29T14:06:54.380679+0000","last_clean":"2023-03-29T14:06:54.380679+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:06:54.380679+0000","last_undegraded":"2023-03-29T14:06:54.380679+0000","last_fullsized":"2023-03-29T14:06:54.380679+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":88,"num_read_kb":49143,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":213,"reported_epoch":268,"state":"active+undersized","last_fresh":"2023-03-29T14:06:53.224569+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:53.224569+0000","last_peered":"2023-03-29T14:06:53.224569+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:53.224569+0000","last_undegraded":"2023-03-29T14:06:53.224569+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'63","reported_seq":131,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:53.380534+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:06:53.380534+0000","last_peered":"2023-03-29T14:06:53.380534+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:53.380534+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210500,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":228,"reported_epoch":269,"state":"active+undersized","last_fresh":"2023-03-29T14:06:54.376303+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:06:54.376303+0000","last_peered":"2023-03-29T14:06:54.376303+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:06:54.376303+0000","last_undegraded":"2023-03-29T14:06:54.376303+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":399,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:53.386494+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:53.386494+0000","last_peered":"2023-03-29T14:06:53.386494+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:53.386494+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":74,"reported_epoch":268,"state":"active+undersized","last_fresh":"2023-03-29T14:06:53.228706+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:53.228706+0000","last_peered":"2023-03-29T14:06:53.228706+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:53.228706+0000","last_undegraded":"2023-03-29T14:06:53.228706+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"268'96","reported_seq":417,"reported_epoch":268,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:54.643532+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:54.643532+0000","last_peered":"2023-03-29T14:06:54.643532+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:54.643532+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6319630,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":113,"num_read_kb":50874,"num_write":188,"num_write_kb":56076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":228,"reported_epoch":269,"state":"active+undersized","last_fresh":"2023-03-29T14:06:54.387241+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:06:54.387241+0000","last_peered":"2023-03-29T14:06:54.387241+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:06:54.387241+0000","last_undegraded":"2023-03-29T14:06:54.387241+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":438,"reported_epoch":268,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:53.220340+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:53.220340+0000","last_peered":"2023-03-29T14:06:53.220340+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:53.220340+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"261'115","reported_seq":535,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:54.366703+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:06:54.366703+0000","last_peered":"2023-03-29T14:06:54.366703+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:06:54.366703+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23295228,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":258,"num_read_kb":179138,"num_write":232,"num_write_kb":66568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":228,"reported_epoch":269,"state":"active+undersized","last_fresh":"2023-03-29T14:06:54.386470+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:06:54.386470+0000","last_peered":"2023-03-29T14:06:54.386470+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:06:54.386470+0000","last_undegraded":"2023-03-29T14:06:54.386470+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"269'226","reported_seq":748,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:55.491666+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:55.491666+0000","last_peered":"2023-03-29T14:06:55.491666+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:55.491666+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33278982,"num_objects":76,"num_object_clones":62,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":385,"num_read_kb":238803,"num_write":426,"num_write_kb":122068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":76}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":213,"reported_epoch":268,"state":"active+undersized","last_fresh":"2023-03-29T14:06:53.242325+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:53.242325+0000","last_peered":"2023-03-29T14:06:53.242325+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:53.242325+0000","last_undegraded":"2023-03-29T14:06:53.242325+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"249'75","reported_seq":383,"reported_epoch":268,"state":"active+clean","last_fresh":"2023-03-29T14:06:53.232610+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:53.232610+0000","last_peered":"2023-03-29T14:06:53.232610+0000","last_clean":"2023-03-29T14:06:53.232610+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:53.232610+0000","last_undegraded":"2023-03-29T14:06:53.232610+0000","last_fullsized":"2023-03-29T14:06:53.232610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4470794,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":70176,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"268'167","reported_seq":566,"reported_epoch":268,"state":"active+clean","last_fresh":"2023-03-29T14:06:54.606044+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:54.606044+0000","last_peered":"2023-03-29T14:06:54.606044+0000","last_clean":"2023-03-29T14:06:54.606044+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:54.606044+0000","last_undegraded":"2023-03-29T14:06:54.606044+0000","last_fullsized":"2023-03-29T14:06:54.606044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773914,"num_objects":58,"num_object_clones":48,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":205,"num_read_kb":115325,"num_write":314,"num_write_kb":92608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":213,"reported_epoch":268,"state":"active+undersized","last_fresh":"2023-03-29T14:06:53.251328+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:53.251328+0000","last_peered":"2023-03-29T14:06:53.251328+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:53.251328+0000","last_undegraded":"2023-03-29T14:06:53.251328+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":228,"reported_epoch":269,"state":"active+undersized","last_fresh":"2023-03-29T14:06:54.381745+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:06:54.381745+0000","last_peered":"2023-03-29T14:06:54.381745+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:06:54.381745+0000","last_undegraded":"2023-03-29T14:06:54.381745+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"263'154","reported_seq":555,"reported_epoch":269,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:53.372702+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:53.372702+0000","last_peered":"2023-03-29T14:06:53.372702+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:53.372702+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19294964,"num_objects":54,"num_object_clones":46,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":203,"num_read_kb":119529,"num_write":300,"num_write_kb":90696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":54}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":193656722,"num_objects":598,"num_object_clones":506,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":2426,"num_read_kb":1438094,"num_write":3238,"num_write_kb":937902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1743,"ondisk_log_size":1743,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":331344,"kb_used_data":316104,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94040496,"statfs":{"total":96636764160,"available":96297467904,"internally_reserved":0,"allocated":323690496,"data_stored":386061139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":375628,"kb_used_data":359236,"kb_used_omap":0,"kb_used_meta":16384,"kb_avail":93996212,"statfs":{"total":96636764160,"available":96252121088,"internally_reserved":0,"allocated":367857664,"data_stored":444866669,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16777216},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":340172,"kb_used_data":324356,"kb_used_omap":0,"kb_used_meta":15808,"kb_avail":94031668,"statfs":{"total":96636764160,"available":96288428032,"internally_reserved":0,"allocated":332140544,"data_stored":401216773,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16187392},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:06:57.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.e 3.9 3.5 3.2 3.1 3.6 2023-03-29T14:06:57.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:57.642+0000 7f8fa7ebf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:57.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:57.656+0000 7f8fa7ebf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:57.860 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.5,3.6] on osd.0 to force-backfill; instructing pg(s) [3.e] on osd.1 to force-backfill; instructing pg(s) [3.2,3.9,3.f] on osd.3 to force-backfill; 2023-03-29T14:06:57.860 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:06:58.110 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:06:58.228 INFO:tasks.rados.rados.0.smithi017.stdout:1117: done (3 left) 2023-03-29T14:06:58.228 INFO:tasks.rados.rados.0.smithi017.stdout:1118: snap_create 2023-03-29T14:06:58.228 INFO:tasks.rados.rados.0.smithi017.stdout:1116: expect (ObjNum 521 snap 116 seq_num 521) 2023-03-29T14:06:58.250 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:58.249+0000 7f346fff5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:58.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:06:58.254+0000 7f346fff5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:06:58.341 INFO:tasks.rados.rados.0.smithi017.stdout:1115: finishing write tid 1 to smithi017107884-12 2023-03-29T14:06:58.341 INFO:tasks.rados.rados.0.smithi017.stdout:1115: finishing write tid 2 to smithi017107884-12 2023-03-29T14:06:58.341 INFO:tasks.rados.rados.0.smithi017.stdout:1115: finishing write tid 3 to smithi017107884-12 2023-03-29T14:06:58.341 INFO:tasks.rados.rados.0.smithi017.stdout:1115: finishing write tid 4 to smithi017107884-12 2023-03-29T14:06:58.341 INFO:tasks.rados.rados.0.smithi017.stdout:1115: finishing write tid 5 to smithi017107884-12 2023-03-29T14:06:58.341 INFO:tasks.rados.rados.0.smithi017.stdout:1115: oid 12 updating version 0 to 172 2023-03-29T14:06:58.341 INFO:tasks.rados.rados.0.smithi017.stdout:1115: oid 12 version 172 is already newer than 172 2023-03-29T14:06:58.342 INFO:tasks.rados.rados.0.smithi017.stdout:1115: oid 12 version 172 is already newer than 172 2023-03-29T14:06:58.342 INFO:tasks.rados.rados.0.smithi017.stdout:1115: oid 12 version 172 is already newer than 172 2023-03-29T14:06:58.342 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 172 (ObjNum 547 snap 125 seq_num 547) dirty exists 2023-03-29T14:06:58.342 INFO:tasks.rados.rados.0.smithi017.stdout:1115: left oid 12 (ObjNum 547 snap 125 seq_num 547) 2023-03-29T14:06:58.355 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:06:58.358 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:06:58.359 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:06:59.232 INFO:tasks.rados.rados.0.smithi017.stdout:1114: done (3 left) 2023-03-29T14:06:59.232 INFO:tasks.rados.rados.0.smithi017.stdout:1115: done (2 left) 2023-03-29T14:06:59.232 INFO:tasks.rados.rados.0.smithi017.stdout:1116: done (1 left) 2023-03-29T14:06:59.232 INFO:tasks.rados.rados.0.smithi017.stdout:1118: done (0 left) 2023-03-29T14:06:59.232 INFO:tasks.rados.rados.0.smithi017.stdout:1119: write oid 1 current snap is 126 2023-03-29T14:06:59.232 INFO:tasks.rados.rados.0.smithi017.stdout:1119: seq_num 548 ranges {410236=413868,1234250=729187,2715800=423063} 2023-03-29T14:06:59.237 INFO:tasks.rados.rados.0.smithi017.stdout:1119: writing smithi017107884-1 from 410236 to 824104 tid 1 2023-03-29T14:06:59.246 INFO:tasks.rados.rados.0.smithi017.stdout:1119: writing smithi017107884-1 from 1234250 to 1963437 tid 2 2023-03-29T14:06:59.255 INFO:tasks.rados.rados.0.smithi017.stdout:1119: writing smithi017107884-1 from 2715800 to 3138863 tid 3 2023-03-29T14:06:59.255 INFO:tasks.rados.rados.0.smithi017.stdout:1120: delete oid 30 current snap is 126 2023-03-29T14:06:59.372 INFO:tasks.rados.rados.0.smithi017.stdout:1119: finishing write tid 1 to smithi017107884-1 2023-03-29T14:06:59.380 INFO:tasks.rados.rados.0.smithi017.stdout:1119: finishing write tid 2 to smithi017107884-1 2023-03-29T14:06:59.390 INFO:tasks.rados.rados.0.smithi017.stdout:1119: finishing write tid 3 to smithi017107884-1 2023-03-29T14:06:59.399 INFO:tasks.rados.rados.0.smithi017.stdout:1119: finishing write tid 4 to smithi017107884-1 2023-03-29T14:06:59.401 INFO:tasks.rados.rados.0.smithi017.stdout:1119: finishing write tid 5 to smithi017107884-1 2023-03-29T14:06:59.401 INFO:tasks.rados.rados.0.smithi017.stdout:1119: oid 1 updating version 0 to 159 2023-03-29T14:06:59.401 INFO:tasks.rados.rados.0.smithi017.stdout:1119: oid 1 version 159 is already newer than 159 2023-03-29T14:06:59.402 INFO:tasks.rados.rados.0.smithi017.stdout:1119: oid 1 version 159 is already newer than 159 2023-03-29T14:06:59.402 INFO:tasks.rados.rados.0.smithi017.stdout:1119: oid 1 version 159 is already newer than 159 2023-03-29T14:06:59.402 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 159 (ObjNum 548 snap 126 seq_num 548) dirty exists 2023-03-29T14:06:59.402 INFO:tasks.rados.rados.0.smithi017.stdout:1119: left oid 1 (ObjNum 548 snap 126 seq_num 548) 2023-03-29T14:06:59.426 INFO:tasks.rados.rados.0.smithi017.stdout:1119: done (1 left) 2023-03-29T14:06:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:1120: done (0 left) 2023-03-29T14:06:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:1121: read oid 40 snap -1 2023-03-29T14:06:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:1121: expect (ObjNum 542 snap 123 seq_num 542) 2023-03-29T14:06:59.427 INFO:tasks.rados.rados.0.smithi017.stdout:1122: snap_remove snap 124 2023-03-29T14:07:00.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~110s 2023-03-29T14:07:00.360 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:00.375 INFO:tasks.rados.rados.0.smithi017.stdout:1122: done (1 left) 2023-03-29T14:07:00.375 INFO:tasks.rados.rados.0.smithi017.stdout:1123: snap_create 2023-03-29T14:07:00.376 INFO:tasks.rados.rados.0.smithi017.stdout:1121: expect (ObjNum 542 snap 123 seq_num 542) 2023-03-29T14:07:00.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:00.489+0000 7f2228d3b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:00.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:00.494+0000 7f2228d3b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:00.558 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:00.561 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:00.561 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:01.377 INFO:tasks.rados.rados.0.smithi017.stdout:1121: done (1 left) 2023-03-29T14:07:01.377 INFO:tasks.rados.rados.0.smithi017.stdout:1123: done (0 left) 2023-03-29T14:07:01.377 INFO:tasks.rados.rados.0.smithi017.stdout:1124: write oid 48 current snap is 127 2023-03-29T14:07:01.377 INFO:tasks.rados.rados.0.smithi017.stdout:1124: seq_num 550 ranges {443850=564366,1634861=732856,2770646=473204} 2023-03-29T14:07:01.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: writing smithi017107884-48 from 443850 to 1008216 tid 1 2023-03-29T14:07:01.394 INFO:tasks.rados.rados.0.smithi017.stdout:1124: writing smithi017107884-48 from 1634861 to 2367717 tid 2 2023-03-29T14:07:01.401 INFO:tasks.rados.rados.0.smithi017.stdout:1124: writing smithi017107884-48 from 2770646 to 3243850 tid 3 2023-03-29T14:07:01.402 INFO:tasks.rados.rados.0.smithi017.stdout:1125: snap_remove snap 126 2023-03-29T14:07:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:1125: done (1 left) 2023-03-29T14:07:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:1126: read oid 12 snap 125 2023-03-29T14:07:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:1126: expect (ObjNum 547 snap 125 seq_num 547) 2023-03-29T14:07:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:1127: write oid 38 current snap is 127 2023-03-29T14:07:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:1124: finishing write tid 1 to smithi017107884-48 2023-03-29T14:07:02.383 INFO:tasks.rados.rados.0.smithi017.stdout:1124: finishing write tid 2 to smithi017107884-48 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: finishing write tid 3 to smithi017107884-48 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: finishing write tid 4 to smithi017107884-48 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: finishing write tid 5 to smithi017107884-48 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: oid 48 updating version 0 to 163 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: oid 48 version 163 is already newer than 163 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: oid 48 version 163 is already newer than 163 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: oid 48 version 163 is already newer than 163 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 163 (ObjNum 550 snap 127 seq_num 550) dirty exists 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1124: left oid 48 (ObjNum 550 snap 127 seq_num 550) 2023-03-29T14:07:02.384 INFO:tasks.rados.rados.0.smithi017.stdout:1127: seq_num 551 ranges {460657=639615,1693343=476514,2681716=578941} 2023-03-29T14:07:02.391 INFO:tasks.rados.rados.0.smithi017.stdout:1127: writing smithi017107884-38 from 460657 to 1100272 tid 1 2023-03-29T14:07:02.399 INFO:tasks.rados.rados.0.smithi017.stdout:1127: writing smithi017107884-38 from 1693343 to 2169857 tid 2 2023-03-29T14:07:02.407 INFO:tasks.rados.rados.0.smithi017.stdout:1127: writing smithi017107884-38 from 2681716 to 3260657 tid 3 2023-03-29T14:07:02.407 INFO:tasks.rados.rados.0.smithi017.stdout:1124: done (2 left) 2023-03-29T14:07:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1128: delete oid 11 current snap is 127 2023-03-29T14:07:02.470 INFO:tasks.rados.rados.0.smithi017.stdout:1128: done (2 left) 2023-03-29T14:07:02.470 INFO:tasks.rados.rados.0.smithi017.stdout:1129: snap_remove snap 122 2023-03-29T14:07:02.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:02.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:02.696+0000 7f2d05806700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:02.700+0000 7f2d05806700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:02.768 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:02.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:02.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:02.872 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:02.872 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:02.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:03.002+0000 7f612a536700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:03.008 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:03.008+0000 7f612a536700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:03.210 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:03.212 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:03.224 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2023-03-29T14:07:02.381744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191610250,"num_objects":610,"num_object_clones":514,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":2586,"num_read_kb":1445893,"num_write":3542,"num_write_kb":950784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1854,"ondisk_log_size":1854,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1210236,"kb_used_data":1150556,"kb_used_omap":0,"kb_used_meta":59648,"kb_avail":376277124,"statfs":{"total":386547056640,"available":385307774976,"internally_reserved":0,"allocated":1178169344,"data_stored":1416535839,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":61079552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3227208,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":18,"num_read_kb":7679,"num_write":30,"num_write_kb":9094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.155105"},"pg_stats":[{"pgid":"3.f","version":"264'83","reported_seq":191,"reported_epoch":275,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:00.383978+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:00.383978+0000","last_peered":"2023-03-29T14:07:00.383978+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:00.383978+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698068,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":28059,"num_write":88,"num_write_kb":25012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":10}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"256'32","reported_seq":284,"reported_epoch":272,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:57.425268+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:06:57.425268+0000","last_peered":"2023-03-29T14:06:57.425268+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:06:57.425268+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5580122,"num_objects":14,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":25430,"num_write":62,"num_write_kb":18414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":14}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"275'162","reported_seq":584,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-03-29T14:07:01.433953+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:01.433953+0000","last_peered":"2023-03-29T14:07:01.433953+0000","last_clean":"2023-03-29T14:07:01.433953+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:01.433953+0000","last_undegraded":"2023-03-29T14:07:01.433953+0000","last_fullsized":"2023-03-29T14:07:01.433953+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17210962,"num_objects":56,"num_object_clones":46,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":224,"num_read_kb":143080,"num_write":308,"num_write_kb":90844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"268'157","reported_seq":585,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-03-29T14:07:00.395662+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:00.395662+0000","last_peered":"2023-03-29T14:07:00.395662+0000","last_clean":"2023-03-29T14:07:00.395662+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:00.395662+0000","last_undegraded":"2023-03-29T14:07:00.395662+0000","last_fullsized":"2023-03-29T14:07:00.395662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244576,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":243,"num_read_kb":145089,"num_write":308,"num_write_kb":86580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"264'165","reported_seq":573,"reported_epoch":272,"state":"active+clean","last_fresh":"2023-03-29T14:06:57.401779+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:06:57.401779+0000","last_peered":"2023-03-29T14:06:57.401779+0000","last_clean":"2023-03-29T14:06:57.401779+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:06:57.401779+0000","last_undegraded":"2023-03-29T14:06:57.401779+0000","last_fullsized":"2023-03-29T14:06:57.401779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15832788,"num_objects":62,"num_object_clones":54,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":209,"num_read_kb":115084,"num_write":334,"num_write_kb":96858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":230,"reported_epoch":275,"state":"active+undersized","last_fresh":"2023-03-29T14:07:00.381444+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:00.381444+0000","last_peered":"2023-03-29T14:07:00.381444+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:00.381444+0000","last_undegraded":"2023-03-29T14:07:00.381444+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"260'104","reported_seq":452,"reported_epoch":275,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:00.397390+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:00.397390+0000","last_peered":"2023-03-29T14:07:00.397390+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:00.397390+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748192,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":127,"num_read_kb":61239,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'69","reported_seq":380,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-03-29T14:07:00.390043+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:00.390043+0000","last_peered":"2023-03-29T14:07:00.390043+0000","last_clean":"2023-03-29T14:07:00.390043+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:00.390043+0000","last_undegraded":"2023-03-29T14:07:00.390043+0000","last_fullsized":"2023-03-29T14:07:00.390043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":52821,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":216,"reported_epoch":272,"state":"active+undersized","last_fresh":"2023-03-29T14:06:57.421584+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:06:57.421584+0000","last_peered":"2023-03-29T14:06:57.421584+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:06:57.421584+0000","last_undegraded":"2023-03-29T14:06:57.421584+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'63","reported_seq":136,"reported_epoch":273,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:59.378184+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:06:59.378184+0000","last_peered":"2023-03-29T14:06:59.378184+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:06:59.378184+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":233,"reported_epoch":275,"state":"active+undersized","last_fresh":"2023-03-29T14:07:00.385289+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:00.385289+0000","last_peered":"2023-03-29T14:07:00.385289+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:00.385289+0000","last_undegraded":"2023-03-29T14:07:00.385289+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":402,"reported_epoch":273,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:58.275289+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:06:58.275289+0000","last_peered":"2023-03-29T14:06:58.275289+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:06:58.275289+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":272,"state":"active+undersized","last_fresh":"2023-03-29T14:06:57.443470+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:06:57.443470+0000","last_peered":"2023-03-29T14:06:57.443470+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:06:57.443470+0000","last_undegraded":"2023-03-29T14:06:57.443470+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"268'96","reported_seq":420,"reported_epoch":272,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:57.398659+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:06:57.398659+0000","last_peered":"2023-03-29T14:06:57.398659+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:06:57.398659+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6319630,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":113,"num_read_kb":50874,"num_write":188,"num_write_kb":56076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":233,"reported_epoch":275,"state":"active+undersized","last_fresh":"2023-03-29T14:07:00.391155+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:00.391155+0000","last_peered":"2023-03-29T14:07:00.391155+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:00.391155+0000","last_undegraded":"2023-03-29T14:07:00.391155+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":441,"reported_epoch":272,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:57.418516+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:06:57.418516+0000","last_peered":"2023-03-29T14:06:57.418516+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:06:57.418516+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"261'115","reported_seq":540,"reported_epoch":275,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:00.393832+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:00.393832+0000","last_peered":"2023-03-29T14:07:00.393832+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:00.393832+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23295228,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":258,"num_read_kb":179138,"num_write":232,"num_write_kb":66568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":233,"reported_epoch":275,"state":"active+undersized","last_fresh":"2023-03-29T14:07:00.388734+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:00.388734+0000","last_peered":"2023-03-29T14:07:00.388734+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:00.388734+0000","last_undegraded":"2023-03-29T14:07:00.388734+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"269'226","reported_seq":751,"reported_epoch":273,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:58.252812+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:06:58.252812+0000","last_peered":"2023-03-29T14:06:58.252812+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:06:58.252812+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33278982,"num_objects":76,"num_object_clones":62,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":385,"num_read_kb":238803,"num_write":426,"num_write_kb":122068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":76}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":272,"state":"active+undersized","last_fresh":"2023-03-29T14:06:57.413332+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:06:57.413332+0000","last_peered":"2023-03-29T14:06:57.413332+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:06:57.413332+0000","last_undegraded":"2023-03-29T14:06:57.413332+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"249'75","reported_seq":386,"reported_epoch":272,"state":"active+clean","last_fresh":"2023-03-29T14:06:57.441860+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:06:57.441860+0000","last_peered":"2023-03-29T14:06:57.441860+0000","last_clean":"2023-03-29T14:06:57.441860+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:06:57.441860+0000","last_undegraded":"2023-03-29T14:06:57.441860+0000","last_fullsized":"2023-03-29T14:06:57.441860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4470794,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":70176,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"271'172","reported_seq":578,"reported_epoch":272,"state":"active+clean","last_fresh":"2023-03-29T14:06:57.518877+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:06:57.518877+0000","last_peered":"2023-03-29T14:06:57.518877+0000","last_clean":"2023-03-29T14:06:57.518877+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:06:57.518877+0000","last_undegraded":"2023-03-29T14:06:57.518877+0000","last_fullsized":"2023-03-29T14:06:57.518877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051312,"num_objects":60,"num_object_clones":50,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":206,"num_read_kb":115326,"num_write":324,"num_write_kb":95176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":216,"reported_epoch":272,"state":"active+undersized","last_fresh":"2023-03-29T14:06:57.423435+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:06:57.423435+0000","last_peered":"2023-03-29T14:06:57.423435+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:06:57.423435+0000","last_undegraded":"2023-03-29T14:06:57.423435+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":275,"state":"active+undersized","last_fresh":"2023-03-29T14:07:00.393099+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:00.393099+0000","last_peered":"2023-03-29T14:07:00.393099+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:00.393099+0000","last_undegraded":"2023-03-29T14:07:00.393099+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"273'159","reported_seq":567,"reported_epoch":273,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:06:59.365086+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:06:59.365086+0000","last_peered":"2023-03-29T14:06:59.365086+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:06:59.365086+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19765560,"num_objects":56,"num_object_clones":48,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":204,"num_read_kb":119530,"num_write":310,"num_write_kb":93760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":56}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":190429514,"num_objects":606,"num_object_clones":514,"num_object_copies":1818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":2444,"num_read_kb":1445773,"num_write":3268,"num_write_kb":946996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1757,"ondisk_log_size":1757,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":332660,"kb_used_data":317420,"kb_used_omap":0,"kb_used_meta":15232,"kb_avail":94039180,"statfs":{"total":96636764160,"available":96296120320,"internally_reserved":0,"allocated":325038080,"data_stored":388151372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15597568},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705958,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":378544,"kb_used_data":362088,"kb_used_omap":0,"kb_used_meta":16448,"kb_avail":93993296,"statfs":{"total":96636764160,"available":96249135104,"internally_reserved":0,"allocated":370778112,"data_stored":449401063,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16842752},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705958,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":343088,"kb_used_data":327208,"kb_used_omap":0,"kb_used_meta":15872,"kb_avail":94028752,"statfs":{"total":96636764160,"available":96285442048,"internally_reserved":0,"allocated":335060992,"data_stored":405751179,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16252928},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:03.386 INFO:tasks.rados.rados.0.smithi017.stdout:1129: done (2 left) 2023-03-29T14:07:03.386 INFO:tasks.rados.rados.0.smithi017.stdout:1130: write (excl) oid 22 current snap is 127 2023-03-29T14:07:03.387 INFO:tasks.rados.rados.0.smithi017.stdout:1130: seq_num 553 ranges {494271=790113,1693954=480183,2620209=512129} 2023-03-29T14:07:03.396 INFO:tasks.rados.rados.0.smithi017.stdout:1130: writing smithi017107884-22 from 494271 to 1284384 tid 1 2023-03-29T14:07:03.403 INFO:tasks.rados.rados.0.smithi017.stdout:1130: writing smithi017107884-22 from 1693954 to 2174137 tid 2 2023-03-29T14:07:03.410 INFO:tasks.rados.rados.0.smithi017.stdout:1130: writing smithi017107884-22 from 2620209 to 3132338 tid 3 2023-03-29T14:07:03.410 INFO:tasks.rados.rados.0.smithi017.stdout:1131: read oid 5 snap 112 2023-03-29T14:07:03.410 INFO:tasks.rados.rados.0.smithi017.stdout:1131: expect (ObjNum 499 snap 112 seq_num 499) 2023-03-29T14:07:03.410 INFO:tasks.rados.rados.0.smithi017.stdout:1132: snap_create 2023-03-29T14:07:03.410 INFO:tasks.rados.rados.0.smithi017.stdout:1127: finishing write tid 1 to smithi017107884-38 2023-03-29T14:07:03.410 INFO:tasks.rados.rados.0.smithi017.stdout:1126: expect (ObjNum 547 snap 125 seq_num 547) 2023-03-29T14:07:03.636 INFO:tasks.rados.rados.0.smithi017.stdout:1127: finishing write tid 2 to smithi017107884-38 2023-03-29T14:07:03.636 INFO:tasks.rados.rados.0.smithi017.stdout:1127: finishing write tid 3 to smithi017107884-38 2023-03-29T14:07:03.636 INFO:tasks.rados.rados.0.smithi017.stdout:1127: finishing write tid 4 to smithi017107884-38 2023-03-29T14:07:03.636 INFO:tasks.rados.rados.0.smithi017.stdout:1127: finishing write tid 5 to smithi017107884-38 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:1127: oid 38 updating version 0 to 37 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:1127: oid 38 version 37 is already newer than 36 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:1127: oid 38 version 37 is already newer than 36 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:1127: oid 38 version 37 is already newer than 37 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 37 (ObjNum 551 snap 127 seq_num 551) dirty exists 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:1127: left oid 38 (ObjNum 551 snap 127 seq_num 551) 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:1130: finishing write tid 1 to smithi017107884-22 2023-03-29T14:07:03.637 INFO:tasks.rados.rados.0.smithi017.stdout:1131: expect (ObjNum 499 snap 112 seq_num 499) 2023-03-29T14:07:03.756 INFO:tasks.rados.rados.0.smithi017.stdout:1130: finishing write tid 2 to smithi017107884-22 2023-03-29T14:07:03.756 INFO:tasks.rados.rados.0.smithi017.stdout:1130: finishing write tid 3 to smithi017107884-22 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:1130: finishing write tid 4 to smithi017107884-22 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:1130: finishing write tid 5 to smithi017107884-22 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:1130: oid 22 updating version 0 to 88 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:1130: oid 22 version 88 is already newer than 84 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:1130: oid 22 version 88 is already newer than 88 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:1130: oid 22 version 88 is already newer than 88 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 88 (ObjNum 553 snap 127 seq_num 553) dirty exists 2023-03-29T14:07:03.757 INFO:tasks.rados.rados.0.smithi017.stdout:1130: left oid 22 (ObjNum 553 snap 127 seq_num 553) 2023-03-29T14:07:04.390 INFO:tasks.rados.rados.0.smithi017.stdout:1126: done (4 left) 2023-03-29T14:07:04.391 INFO:tasks.rados.rados.0.smithi017.stdout:1127: done (3 left) 2023-03-29T14:07:04.391 INFO:tasks.rados.rados.0.smithi017.stdout:1130: done (2 left) 2023-03-29T14:07:04.391 INFO:tasks.rados.rados.0.smithi017.stdout:1131: done (1 left) 2023-03-29T14:07:04.391 INFO:tasks.rados.rados.0.smithi017.stdout:1132: done (0 left) 2023-03-29T14:07:04.391 INFO:tasks.rados.rados.0.smithi017.stdout:1133: snap_create 2023-03-29T14:07:04.772 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:04.901 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:04.900+0000 7efdebc0f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:04.904+0000 7efdebc0f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:04.968 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:04.971 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:04.972 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:05.391 INFO:tasks.rados.rados.0.smithi017.stdout:1133: done (0 left) 2023-03-29T14:07:05.391 INFO:tasks.rados.rados.0.smithi017.stdout:1134: write oid 36 current snap is 129 2023-03-29T14:07:05.391 INFO:tasks.rados.rados.0.smithi017.stdout:1134: seq_num 554 ranges {511078=465362,1752436=740194,3163985=147093} 2023-03-29T14:07:05.398 INFO:tasks.rados.rados.0.smithi017.stdout:1134: writing smithi017107884-36 from 511078 to 976440 tid 1 2023-03-29T14:07:05.409 INFO:tasks.rados.rados.0.smithi017.stdout:1134: writing smithi017107884-36 from 1752436 to 2492630 tid 2 2023-03-29T14:07:05.415 INFO:tasks.rados.rados.0.smithi017.stdout:1134: writing smithi017107884-36 from 3163985 to 3311078 tid 3 2023-03-29T14:07:05.415 INFO:tasks.rados.rados.0.smithi017.stdout:1135: write oid 5 current snap is 129 2023-03-29T14:07:05.415 INFO:tasks.rados.rados.0.smithi017.stdout:1135: seq_num 555 ranges {527885=656964,1927271=483852,3191408=136477} 2023-03-29T14:07:05.424 INFO:tasks.rados.rados.0.smithi017.stdout:1135: writing smithi017107884-5 from 527885 to 1184849 tid 1 2023-03-29T14:07:05.433 INFO:tasks.rados.rados.0.smithi017.stdout:1135: writing smithi017107884-5 from 1927271 to 2411123 tid 2 2023-03-29T14:07:05.438 INFO:tasks.rados.rados.0.smithi017.stdout:1135: writing smithi017107884-5 from 3191408 to 3327885 tid 3 2023-03-29T14:07:05.439 INFO:tasks.rados.rados.0.smithi017.stdout:1136: snap_create 2023-03-29T14:07:05.550 INFO:tasks.rados.rados.0.smithi017.stdout:1134: finishing write tid 1 to smithi017107884-36 2023-03-29T14:07:05.561 INFO:tasks.rados.rados.0.smithi017.stdout:1134: finishing write tid 2 to smithi017107884-36 2023-03-29T14:07:05.596 INFO:tasks.rados.rados.0.smithi017.stdout:1134: finishing write tid 3 to smithi017107884-36 2023-03-29T14:07:05.608 INFO:tasks.rados.rados.0.smithi017.stdout:1134: finishing write tid 4 to smithi017107884-36 2023-03-29T14:07:05.621 INFO:tasks.rados.rados.0.smithi017.stdout:1134: finishing write tid 5 to smithi017107884-36 2023-03-29T14:07:05.622 INFO:tasks.rados.rados.0.smithi017.stdout:1134: oid 36 updating version 0 to 100 2023-03-29T14:07:05.622 INFO:tasks.rados.rados.0.smithi017.stdout:1134: oid 36 version 100 is already newer than 100 2023-03-29T14:07:05.622 INFO:tasks.rados.rados.0.smithi017.stdout:1134: oid 36 version 100 is already newer than 100 2023-03-29T14:07:05.622 INFO:tasks.rados.rados.0.smithi017.stdout:1134: oid 36 version 100 is already newer than 100 2023-03-29T14:07:05.622 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 100 (ObjNum 554 snap 129 seq_num 554) dirty exists 2023-03-29T14:07:05.622 INFO:tasks.rados.rados.0.smithi017.stdout:1134: left oid 36 (ObjNum 554 snap 129 seq_num 554) 2023-03-29T14:07:05.623 INFO:tasks.rados.rados.0.smithi017.stdout:1135: finishing write tid 1 to smithi017107884-5 2023-03-29T14:07:05.625 INFO:tasks.rados.rados.0.smithi017.stdout:1135: finishing write tid 2 to smithi017107884-5 2023-03-29T14:07:05.628 INFO:tasks.rados.rados.0.smithi017.stdout:1135: finishing write tid 3 to smithi017107884-5 2023-03-29T14:07:05.636 INFO:tasks.rados.rados.0.smithi017.stdout:1135: finishing write tid 4 to smithi017107884-5 2023-03-29T14:07:05.638 INFO:tasks.rados.rados.0.smithi017.stdout:1135: finishing write tid 5 to smithi017107884-5 2023-03-29T14:07:05.638 INFO:tasks.rados.rados.0.smithi017.stdout:1135: oid 5 updating version 0 to 177 2023-03-29T14:07:05.639 INFO:tasks.rados.rados.0.smithi017.stdout:1135: oid 5 version 177 is already newer than 177 2023-03-29T14:07:05.639 INFO:tasks.rados.rados.0.smithi017.stdout:1135: oid 5 version 177 is already newer than 177 2023-03-29T14:07:05.639 INFO:tasks.rados.rados.0.smithi017.stdout:1135: oid 5 version 177 is already newer than 177 2023-03-29T14:07:05.639 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 177 (ObjNum 555 snap 129 seq_num 555) dirty exists 2023-03-29T14:07:05.639 INFO:tasks.rados.rados.0.smithi017.stdout:1135: left oid 5 (ObjNum 555 snap 129 seq_num 555) 2023-03-29T14:07:05.651 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~116s 2023-03-29T14:07:06.397 INFO:tasks.rados.rados.0.smithi017.stdout:1134: done (2 left) 2023-03-29T14:07:06.397 INFO:tasks.rados.rados.0.smithi017.stdout:1135: done (1 left) 2023-03-29T14:07:06.397 INFO:tasks.rados.rados.0.smithi017.stdout:1136: done (0 left) 2023-03-29T14:07:06.397 INFO:tasks.rados.rados.0.smithi017.stdout:1137: write oid 34 current snap is 130 2023-03-29T14:07:06.397 INFO:tasks.rados.rados.0.smithi017.stdout:1137: seq_num 556 ranges {544692=732213,1985753=743863,3335184=9508} 2023-03-29T14:07:06.407 INFO:tasks.rados.rados.0.smithi017.stdout:1137: writing smithi017107884-34 from 544692 to 1276905 tid 1 2023-03-29T14:07:06.418 INFO:tasks.rados.rados.0.smithi017.stdout:1137: writing smithi017107884-34 from 1985753 to 2729616 tid 2 2023-03-29T14:07:06.421 INFO:tasks.rados.rados.0.smithi017.stdout:1137: writing smithi017107884-34 from 3335184 to 3344692 tid 3 2023-03-29T14:07:06.421 INFO:tasks.rados.rados.0.smithi017.stdout:1138: snap_create 2023-03-29T14:07:06.577 INFO:tasks.rados.rados.0.smithi017.stdout:1137: finishing write tid 1 to smithi017107884-34 2023-03-29T14:07:06.579 INFO:tasks.rados.rados.0.smithi017.stdout:1137: finishing write tid 2 to smithi017107884-34 2023-03-29T14:07:06.581 INFO:tasks.rados.rados.0.smithi017.stdout:1137: finishing write tid 3 to smithi017107884-34 2023-03-29T14:07:06.582 INFO:tasks.rados.rados.0.smithi017.stdout:1137: finishing write tid 4 to smithi017107884-34 2023-03-29T14:07:06.585 INFO:tasks.rados.rados.0.smithi017.stdout:1137: finishing write tid 5 to smithi017107884-34 2023-03-29T14:07:06.585 INFO:tasks.rados.rados.0.smithi017.stdout:1137: oid 34 updating version 0 to 170 2023-03-29T14:07:06.585 INFO:tasks.rados.rados.0.smithi017.stdout:1137: oid 34 version 170 is already newer than 170 2023-03-29T14:07:06.585 INFO:tasks.rados.rados.0.smithi017.stdout:1137: oid 34 version 170 is already newer than 170 2023-03-29T14:07:06.585 INFO:tasks.rados.rados.0.smithi017.stdout:1137: oid 34 version 170 is already newer than 170 2023-03-29T14:07:06.586 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 170 (ObjNum 556 snap 130 seq_num 556) dirty exists 2023-03-29T14:07:06.586 INFO:tasks.rados.rados.0.smithi017.stdout:1137: left oid 34 (ObjNum 556 snap 130 seq_num 556) 2023-03-29T14:07:06.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:07.102 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:07.100+0000 7f761a3b4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:07.104 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:07.105+0000 7f761a3b4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:07.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:07.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:07.400 INFO:tasks.rados.rados.0.smithi017.stdout:1137: done (1 left) 2023-03-29T14:07:07.400 INFO:tasks.rados.rados.0.smithi017.stdout:1138: done (0 left) 2023-03-29T14:07:07.400 INFO:tasks.rados.rados.0.smithi017.stdout:1139: snap_remove snap 125 2023-03-29T14:07:08.230 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:08.230 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:08.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:08.231 INFO:tasks.rados.rados.0.smithi017.stdout:1139: done (0 left) 2023-03-29T14:07:08.231 INFO:tasks.rados.rados.0.smithi017.stdout:1140: read oid 44 snap -1 2023-03-29T14:07:08.232 INFO:tasks.rados.rados.0.smithi017.stdout:1140: expect deleted 2023-03-29T14:07:08.232 INFO:tasks.rados.rados.0.smithi017.stdout:1141: write oid 12 current snap is 131 2023-03-29T14:07:08.232 INFO:tasks.rados.rados.0.smithi017.stdout:1141: seq_num 557 ranges {561499=407462,1527882=487521,2729901=557217} 2023-03-29T14:07:08.237 INFO:tasks.rados.rados.0.smithi017.stdout:1141: writing smithi017107884-12 from 561499 to 968961 tid 1 2023-03-29T14:07:08.245 INFO:tasks.rados.rados.0.smithi017.stdout:1141: writing smithi017107884-12 from 1527882 to 2015403 tid 2 2023-03-29T14:07:08.255 INFO:tasks.rados.rados.0.smithi017.stdout:1141: writing smithi017107884-12 from 2729901 to 3287118 tid 3 2023-03-29T14:07:08.255 INFO:tasks.rados.rados.0.smithi017.stdout:1142: read oid 25 snap -1 2023-03-29T14:07:08.255 INFO:tasks.rados.rados.0.smithi017.stdout:1142: expect (ObjNum 507 snap 113 seq_num 507) 2023-03-29T14:07:08.255 INFO:tasks.rados.rados.0.smithi017.stdout:1143: read oid 36 snap -1 2023-03-29T14:07:08.255 INFO:tasks.rados.rados.0.smithi017.stdout:1143: expect (ObjNum 554 snap 129 seq_num 554) 2023-03-29T14:07:08.255 INFO:tasks.rados.rados.0.smithi017.stdout:1144: write oid 9 current snap is 131 2023-03-29T14:07:08.255 INFO:tasks.rados.rados.0.smithi017.stdout:1144: seq_num 558 ranges {578306=482711,1586364=631179,2757324=484842} 2023-03-29T14:07:08.262 INFO:tasks.rados.rados.0.smithi017.stdout:1144: writing smithi017107884-9 from 578306 to 1061017 tid 1 2023-03-29T14:07:08.271 INFO:tasks.rados.rados.0.smithi017.stdout:1144: writing smithi017107884-9 from 1586364 to 2217543 tid 2 2023-03-29T14:07:08.280 INFO:tasks.rados.rados.0.smithi017.stdout:1144: writing smithi017107884-9 from 2757324 to 3242166 tid 3 2023-03-29T14:07:08.280 INFO:tasks.rados.rados.0.smithi017.stdout:1145: read oid 22 snap 129 2023-03-29T14:07:08.280 INFO:tasks.rados.rados.0.smithi017.stdout:1145: expect (ObjNum 553 snap 127 seq_num 553) 2023-03-29T14:07:08.280 INFO:tasks.rados.rados.0.smithi017.stdout:1146: write (excl) oid 15 current snap is 131 2023-03-29T14:07:08.280 INFO:tasks.rados.rados.0.smithi017.stdout:1146: seq_num 559 ranges {595113=557960,1644846=491190,2784747=610366} 2023-03-29T14:07:08.289 INFO:tasks.rados.rados.0.smithi017.stdout:1146: writing smithi017107884-15 from 595113 to 1153073 tid 1 2023-03-29T14:07:08.297 INFO:tasks.rados.rados.0.smithi017.stdout:1146: writing smithi017107884-15 from 1644846 to 2136036 tid 2 2023-03-29T14:07:08.308 INFO:tasks.rados.rados.0.smithi017.stdout:1146: writing smithi017107884-15 from 2784747 to 3395113 tid 3 2023-03-29T14:07:08.308 INFO:tasks.rados.rados.0.smithi017.stdout:1147: read oid 48 snap -1 2023-03-29T14:07:08.308 INFO:tasks.rados.rados.0.smithi017.stdout:1147: expect (ObjNum 550 snap 127 seq_num 550) 2023-03-29T14:07:08.308 INFO:tasks.rados.rados.0.smithi017.stdout:1148: write oid 35 current snap is 131 2023-03-29T14:07:08.308 INFO:tasks.rados.rados.0.smithi017.stdout:1148: seq_num 560 ranges {611920=633209,1703328=634848,2812170=507386} 2023-03-29T14:07:08.317 INFO:tasks.rados.rados.0.smithi017.stdout:1148: writing smithi017107884-35 from 611920 to 1245129 tid 1 2023-03-29T14:07:08.326 INFO:tasks.rados.rados.0.smithi017.stdout:1148: writing smithi017107884-35 from 1703328 to 2338176 tid 2 2023-03-29T14:07:08.333 INFO:tasks.rados.rados.0.smithi017.stdout:1148: writing smithi017107884-35 from 2812170 to 3319556 tid 3 2023-03-29T14:07:08.334 INFO:tasks.rados.rados.0.smithi017.stdout:1149: write oid 20 current snap is 131 2023-03-29T14:07:08.334 INFO:tasks.rados.rados.0.smithi017.stdout:1149: seq_num 561 ranges {628727=708458,2045457=494859,3123240=305487} 2023-03-29T14:07:08.344 INFO:tasks.rados.rados.0.smithi017.stdout:1149: writing smithi017107884-20 from 628727 to 1337185 tid 1 2023-03-29T14:07:08.353 INFO:tasks.rados.rados.0.smithi017.stdout:1149: writing smithi017107884-20 from 2045457 to 2540316 tid 2 2023-03-29T14:07:08.359 INFO:tasks.rados.rados.0.smithi017.stdout:1149: writing smithi017107884-20 from 3123240 to 3428727 tid 3 2023-03-29T14:07:08.359 INFO:tasks.rados.rados.0.smithi017.stdout:1150: delete oid 13 current snap is 131 2023-03-29T14:07:08.359 INFO:tasks.rados.rados.0.smithi017.stdout:1142: expect (ObjNum 507 snap 113 seq_num 507) 2023-03-29T14:07:08.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:08.365+0000 7f2fad438700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:08.372+0000 7f2fad438700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:08.533 INFO:tasks.rados.rados.0.smithi017.stdout:1145: expect (ObjNum 553 snap 127 seq_num 553) 2023-03-29T14:07:08.775 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:08.776 INFO:tasks.rados.rados.0.smithi017.stdout:1147: expect (ObjNum 550 snap 127 seq_num 550) 2023-03-29T14:07:08.778 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:08.790 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2023-03-29T14:07:08.229085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":200988754,"num_objects":616,"num_object_clones":520,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":2607,"num_read_kb":1458440,"num_write":3582,"num_write_kb":962724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1876,"ondisk_log_size":1876,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1225136,"kb_used_data":1165136,"kb_used_omap":0,"kb_used_meta":59968,"kb_avail":376262224,"statfs":{"total":386547056640,"available":385292517376,"internally_reserved":0,"allocated":1193099264,"data_stored":1436337989,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":61407232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9378504,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":21,"num_read_kb":12547,"num_write":40,"num_write_kb":11940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.840152"},"pg_stats":[{"pgid":"3.f","version":"277'88","reported_seq":204,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:05.535260+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:05.535260+0000","last_peered":"2023-03-29T14:07:05.535260+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:05.535260+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12235892,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":28060,"num_write":98,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":297,"reported_epoch":279,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:05.250364+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:05.250364+0000","last_peered":"2023-03-29T14:07:05.250364+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:05.250364+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"275'163","reported_seq":589,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-03-29T14:07:05.532122+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:05.532122+0000","last_peered":"2023-03-29T14:07:05.532122+0000","last_clean":"2023-03-29T14:07:05.532122+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:05.532122+0000","last_undegraded":"2023-03-29T14:07:05.532122+0000","last_fullsized":"2023-03-29T14:07:05.532122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17210962,"num_objects":56,"num_object_clones":46,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":225,"num_read_kb":143081,"num_write":308,"num_write_kb":90844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"268'157","reported_seq":589,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-03-29T14:07:05.526119+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:05.526119+0000","last_peered":"2023-03-29T14:07:05.526119+0000","last_clean":"2023-03-29T14:07:05.526119+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:05.526119+0000","last_undegraded":"2023-03-29T14:07:05.526119+0000","last_fullsized":"2023-03-29T14:07:05.526119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244576,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":243,"num_read_kb":145089,"num_write":308,"num_write_kb":86580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"264'165","reported_seq":577,"reported_epoch":279,"state":"active+clean","last_fresh":"2023-03-29T14:07:05.260482+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:05.260482+0000","last_peered":"2023-03-29T14:07:05.260482+0000","last_clean":"2023-03-29T14:07:05.260482+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:05.260482+0000","last_undegraded":"2023-03-29T14:07:05.260482+0000","last_fullsized":"2023-03-29T14:07:05.260482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15832788,"num_objects":62,"num_object_clones":54,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":209,"num_read_kb":115084,"num_write":334,"num_write_kb":96858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":234,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.525412+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:05.525412+0000","last_peered":"2023-03-29T14:07:05.525412+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:05.525412+0000","last_undegraded":"2023-03-29T14:07:05.525412+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"260'104","reported_seq":456,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:05.537017+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:05.537017+0000","last_peered":"2023-03-29T14:07:05.537017+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:05.537017+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748192,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":127,"num_read_kb":61239,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'69","reported_seq":384,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-03-29T14:07:05.528420+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:05.528420+0000","last_peered":"2023-03-29T14:07:05.528420+0000","last_clean":"2023-03-29T14:07:05.528420+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:05.528420+0000","last_undegraded":"2023-03-29T14:07:05.528420+0000","last_fullsized":"2023-03-29T14:07:05.528420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":52821,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":220,"reported_epoch":279,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.239302+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:05.239302+0000","last_peered":"2023-03-29T14:07:05.239302+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:05.239302+0000","last_undegraded":"2023-03-29T14:07:05.239302+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'65","reported_seq":140,"reported_epoch":279,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:04.465381+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:04.465381+0000","last_peered":"2023-03-29T14:07:04.465381+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:04.465381+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":237,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.535989+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:05.535989+0000","last_peered":"2023-03-29T14:07:05.535989+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:05.535989+0000","last_undegraded":"2023-03-29T14:07:05.535989+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":406,"reported_epoch":279,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:04.455564+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:04.455564+0000","last_peered":"2023-03-29T14:07:04.455564+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:04.455564+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":81,"reported_epoch":279,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.226414+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:05.226414+0000","last_peered":"2023-03-29T14:07:05.226414+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:05.226414+0000","last_undegraded":"2023-03-29T14:07:05.226414+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"279'100","reported_seq":433,"reported_epoch":279,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:05.542799+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:05.542799+0000","last_peered":"2023-03-29T14:07:05.542799+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:05.542799+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12941786,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":114,"num_read_kb":50875,"num_write":198,"num_write_kb":58720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":237,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.527167+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:05.527167+0000","last_peered":"2023-03-29T14:07:05.527167+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:05.527167+0000","last_undegraded":"2023-03-29T14:07:05.527167+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":445,"reported_epoch":279,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:05.252258+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:05.252258+0000","last_peered":"2023-03-29T14:07:05.252258+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:05.252258+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"261'115","reported_seq":544,"reported_epoch":280,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:05.532720+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:05.532720+0000","last_peered":"2023-03-29T14:07:05.532720+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:05.532720+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23295228,"num_objects":38,"num_object_clones":30,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":258,"num_read_kb":179138,"num_write":232,"num_write_kb":66568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":237,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.534004+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:05.534004+0000","last_peered":"2023-03-29T14:07:05.534004+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:05.534004+0000","last_undegraded":"2023-03-29T14:07:05.534004+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"269'226","reported_seq":756,"reported_epoch":279,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:04.463521+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:04.463521+0000","last_peered":"2023-03-29T14:07:04.463521+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:04.463521+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33278982,"num_objects":76,"num_object_clones":62,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":385,"num_read_kb":238803,"num_write":426,"num_write_kb":122068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":76}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":279,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.248520+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:05.248520+0000","last_peered":"2023-03-29T14:07:05.248520+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:05.248520+0000","last_undegraded":"2023-03-29T14:07:05.248520+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"249'75","reported_seq":390,"reported_epoch":279,"state":"active+clean","last_fresh":"2023-03-29T14:07:05.255238+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:05.255238+0000","last_peered":"2023-03-29T14:07:05.255238+0000","last_clean":"2023-03-29T14:07:05.255238+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:05.255238+0000","last_undegraded":"2023-03-29T14:07:05.255238+0000","last_fullsized":"2023-03-29T14:07:05.255238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4470794,"num_objects":26,"num_object_clones":22,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":120,"num_read_kb":70176,"num_write":156,"num_write_kb":44200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"279'177","reported_seq":597,"reported_epoch":279,"state":"active+clean","last_fresh":"2023-03-29T14:07:05.615913+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:05.615913+0000","last_peered":"2023-03-29T14:07:05.615913+0000","last_clean":"2023-03-29T14:07:05.615913+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:05.615913+0000","last_undegraded":"2023-03-29T14:07:05.615913+0000","last_fullsized":"2023-03-29T14:07:05.615913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19328644,"num_objects":62,"num_object_clones":52,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":221,"num_read_kb":127869,"num_write":334,"num_write_kb":97674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":279,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.228270+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:05.228270+0000","last_peered":"2023-03-29T14:07:05.228270+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:05.228270+0000","last_undegraded":"2023-03-29T14:07:05.228270+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":237,"reported_epoch":280,"state":"active+undersized","last_fresh":"2023-03-29T14:07:05.524152+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:05.524152+0000","last_peered":"2023-03-29T14:07:05.524152+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:05.524152+0000","last_undegraded":"2023-03-29T14:07:05.524152+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"273'159","reported_seq":571,"reported_epoch":279,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:04.458516+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:04.458516+0000","last_peered":"2023-03-29T14:07:04.458516+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:04.458516+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19765560,"num_objects":56,"num_object_clones":48,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":204,"num_read_kb":119530,"num_write":310,"num_write_kb":93760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":56}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":199808018,"num_objects":612,"num_object_clones":520,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":2465,"num_read_kb":1458320,"num_write":3308,"num_write_kb":958936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1779,"ondisk_log_size":1779,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":337560,"kb_used_data":322192,"kb_used_omap":0,"kb_used_meta":15360,"kb_avail":94034280,"statfs":{"total":96636764160,"available":96291102720,"internally_reserved":0,"allocated":329924608,"data_stored":394980318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15728640},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":383856,"kb_used_data":367272,"kb_used_omap":0,"kb_used_meta":16576,"kb_avail":93987984,"statfs":{"total":96636764160,"available":96243695616,"internally_reserved":0,"allocated":376086528,"data_stored":456349901,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16973824},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":347776,"kb_used_data":331832,"kb_used_omap":0,"kb_used_meta":15936,"kb_avail":94024064,"statfs":{"total":96636764160,"available":96280641536,"internally_reserved":0,"allocated":339795968,"data_stored":411775545,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16318464},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:08.794 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.e 3.6 2023-03-29T14:07:08.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:08.972+0000 7f98b8eb2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:08.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:08.981+0000 7f98b8eb2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:08.992 INFO:tasks.rados.rados.0.smithi017.stdout:1144: finishing write tid 1 to smithi017107884-9 2023-03-29T14:07:08.992 INFO:tasks.rados.rados.0.smithi017.stdout:1146: finishing write tid 1 to smithi017107884-15 2023-03-29T14:07:08.993 INFO:tasks.rados.rados.0.smithi017.stdout:1140: done (10 left) 2023-03-29T14:07:08.993 INFO:tasks.rados.rados.0.smithi017.stdout:1142: done (9 left) 2023-03-29T14:07:08.993 INFO:tasks.rados.rados.0.smithi017.stdout:1145: done (8 left) 2023-03-29T14:07:08.993 INFO:tasks.rados.rados.0.smithi017.stdout:1147: done (7 left) 2023-03-29T14:07:08.993 INFO:tasks.rados.rados.0.smithi017.stdout:1150: done (6 left) 2023-03-29T14:07:08.993 INFO:tasks.rados.rados.0.smithi017.stdout:1151: snap_remove snap 129 2023-03-29T14:07:09.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:09.194 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.6] on osd.0 to force-backfill; instructing pg(s) [3.e] on osd.1 to force-backfill; instructing pg(s) [3.f] on osd.3 to force-backfill; 2023-03-29T14:07:09.194 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:07:09.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:09.301+0000 7f14a507a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:09.305 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:09.305+0000 7f14a507a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:09.366 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:09.368 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:09.369 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:1151: done (6 left) 2023-03-29T14:07:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:1152: read oid 13 snap 127 2023-03-29T14:07:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:1152: expect (ObjNum 544 snap 123 seq_num 544) 2023-03-29T14:07:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:1153: write oid 39 current snap is 131 2023-03-29T14:07:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:1144: finishing write tid 2 to smithi017107884-9 2023-03-29T14:07:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:1144: finishing write tid 3 to smithi017107884-9 2023-03-29T14:07:09.408 INFO:tasks.rados.rados.0.smithi017.stdout:1153: seq_num 563 ranges {662341=575309,1878774=498528,2894439=457555} 2023-03-29T14:07:09.414 INFO:tasks.rados.rados.0.smithi017.stdout:1153: writing smithi017107884-39 from 662341 to 1237650 tid 1 2023-03-29T14:07:09.421 INFO:tasks.rados.rados.0.smithi017.stdout:1153: writing smithi017107884-39 from 1878774 to 2377302 tid 2 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1153: writing smithi017107884-39 from 2894439 to 3351994 tid 3 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1154: read oid 17 snap -1 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1154: expect deleted 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1155: write (excl) oid 4 current snap is 131 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1146: finishing write tid 2 to smithi017107884-15 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1144: finishing write tid 4 to smithi017107884-9 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1144: finishing write tid 5 to smithi017107884-9 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1144: oid 9 updating version 0 to 162 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1144: oid 9 version 162 is already newer than 162 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1144: oid 9 version 162 is already newer than 162 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1144: oid 9 version 162 is already newer than 162 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 162 (ObjNum 558 snap 131 seq_num 558) dirty exists 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1144: left oid 9 (ObjNum 558 snap 131 seq_num 558) 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1146: finishing write tid 3 to smithi017107884-15 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1146: finishing write tid 4 to smithi017107884-15 2023-03-29T14:07:09.427 INFO:tasks.rados.rados.0.smithi017.stdout:1141: finishing write tid 1 to smithi017107884-12 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1146: finishing write tid 5 to smithi017107884-15 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1146: oid 15 updating version 0 to 228 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1146: oid 15 updating version 228 to 232 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1146: oid 15 version 232 is already newer than 232 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1146: oid 15 version 232 is already newer than 232 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 232 (ObjNum 559 snap 131 seq_num 559) dirty exists 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1146: left oid 15 (ObjNum 559 snap 131 seq_num 559) 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1141: finishing write tid 2 to smithi017107884-12 2023-03-29T14:07:09.428 INFO:tasks.rados.rados.0.smithi017.stdout:1143: expect (ObjNum 554 snap 129 seq_num 554) 2023-03-29T14:07:09.559 INFO:tasks.rados.rados.0.smithi017.stdout:1141: finishing write tid 3 to smithi017107884-12 2023-03-29T14:07:09.559 INFO:tasks.rados.rados.0.smithi017.stdout:1141: finishing write tid 4 to smithi017107884-12 2023-03-29T14:07:09.559 INFO:tasks.rados.rados.0.smithi017.stdout:1155: seq_num 564 ranges {679148=650558,1937256=642186,3321862=157286} 2023-03-29T14:07:09.566 INFO:tasks.rados.rados.0.smithi017.stdout:1155: writing smithi017107884-4 from 679148 to 1329706 tid 1 2023-03-29T14:07:09.573 INFO:tasks.rados.rados.0.smithi017.stdout:1155: writing smithi017107884-4 from 1937256 to 2579442 tid 2 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1155: writing smithi017107884-4 from 3321862 to 3479148 tid 3 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1143: done (9 left) 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1144: done (8 left) 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1146: done (7 left) 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1156: read oid 45 snap -1 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1156: expect (ObjNum 519 snap 116 seq_num 519) 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1157: write oid 27 current snap is 131 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1141: finishing write tid 5 to smithi017107884-12 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1141: oid 12 updating version 0 to 182 2023-03-29T14:07:09.579 INFO:tasks.rados.rados.0.smithi017.stdout:1141: oid 12 version 182 is already newer than 182 2023-03-29T14:07:09.580 INFO:tasks.rados.rados.0.smithi017.stdout:1141: oid 12 version 182 is already newer than 182 2023-03-29T14:07:09.580 INFO:tasks.rados.rados.0.smithi017.stdout:1141: oid 12 version 182 is already newer than 182 2023-03-29T14:07:09.580 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 182 (ObjNum 557 snap 131 seq_num 557) dirty exists 2023-03-29T14:07:09.580 INFO:tasks.rados.rados.0.smithi017.stdout:1141: left oid 12 (ObjNum 557 snap 131 seq_num 557) 2023-03-29T14:07:09.580 INFO:tasks.rados.rados.0.smithi017.stdout:1148: finishing write tid 1 to smithi017107884-35 2023-03-29T14:07:09.580 INFO:tasks.rados.rados.0.smithi017.stdout:1148: finishing write tid 2 to smithi017107884-35 2023-03-29T14:07:09.580 INFO:tasks.rados.rados.0.smithi017.stdout:1148: finishing write tid 3 to smithi017107884-35 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1148: finishing write tid 4 to smithi017107884-35 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1148: finishing write tid 5 to smithi017107884-35 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1148: oid 35 updating version 0 to 80 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1148: oid 35 version 80 is already newer than 80 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1148: oid 35 version 80 is already newer than 80 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1148: oid 35 version 80 is already newer than 80 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 80 (ObjNum 560 snap 131 seq_num 560) dirty exists 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1148: left oid 35 (ObjNum 560 snap 131 seq_num 560) 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: finishing write tid 1 to smithi017107884-20 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: finishing write tid 2 to smithi017107884-20 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: finishing write tid 3 to smithi017107884-20 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: finishing write tid 4 to smithi017107884-20 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: finishing write tid 5 to smithi017107884-20 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: oid 20 updating version 0 to 175 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: oid 20 version 175 is already newer than 175 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: oid 20 version 175 is already newer than 175 2023-03-29T14:07:09.581 INFO:tasks.rados.rados.0.smithi017.stdout:1149: oid 20 version 175 is already newer than 175 2023-03-29T14:07:09.582 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 175 (ObjNum 561 snap 131 seq_num 561) dirty exists 2023-03-29T14:07:09.582 INFO:tasks.rados.rados.0.smithi017.stdout:1149: left oid 20 (ObjNum 561 snap 131 seq_num 561) 2023-03-29T14:07:09.582 INFO:tasks.rados.rados.0.smithi017.stdout:1153: finishing write tid 1 to smithi017107884-39 2023-03-29T14:07:09.582 INFO:tasks.rados.rados.0.smithi017.stdout:1157: seq_num 565 ranges {695955=725807,1879385=502197,2949285=480099} 2023-03-29T14:07:09.586 INFO:tasks.rados.rados.0.smithi017.stdout:1157: writing smithi017107884-27 from 695955 to 1421762 tid 1 2023-03-29T14:07:09.592 INFO:tasks.rados.rados.0.smithi017.stdout:1157: writing smithi017107884-27 from 1879385 to 2381582 tid 2 2023-03-29T14:07:09.600 INFO:tasks.rados.rados.0.smithi017.stdout:1157: writing smithi017107884-27 from 2949285 to 3429384 tid 3 2023-03-29T14:07:09.600 INFO:tasks.rados.rados.0.smithi017.stdout:1141: done (8 left) 2023-03-29T14:07:09.600 INFO:tasks.rados.rados.0.smithi017.stdout:1148: done (7 left) 2023-03-29T14:07:09.600 INFO:tasks.rados.rados.0.smithi017.stdout:1149: done (6 left) 2023-03-29T14:07:09.600 INFO:tasks.rados.rados.0.smithi017.stdout:1158: read oid 42 snap 128 2023-03-29T14:07:09.600 INFO:tasks.rados.rados.0.smithi017.stdout:1158: expect deleted 2023-03-29T14:07:09.600 INFO:tasks.rados.rados.0.smithi017.stdout:1159: snap_remove snap 130 2023-03-29T14:07:10.411 INFO:tasks.rados.rados.0.smithi017.stdout:1159: done (7 left) 2023-03-29T14:07:10.411 INFO:tasks.rados.rados.0.smithi017.stdout:1160: write oid 23 current snap is 131 2023-03-29T14:07:10.411 INFO:tasks.rados.rados.0.smithi017.stdout:1160: seq_num 566 ranges {712762=401056,1537867=645855,2860355=652407} 2023-03-29T14:07:10.417 INFO:tasks.rados.rados.0.smithi017.stdout:1160: writing smithi017107884-23 from 712762 to 1113818 tid 1 2023-03-29T14:07:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1160: writing smithi017107884-23 from 1537867 to 2183722 tid 2 2023-03-29T14:07:10.431 INFO:tasks.rados.rados.0.smithi017.stdout:1160: writing smithi017107884-23 from 2860355 to 3512762 tid 3 2023-03-29T14:07:10.432 INFO:tasks.rados.rados.0.smithi017.stdout:1161: read oid 25 snap -1 2023-03-29T14:07:10.432 INFO:tasks.rados.rados.0.smithi017.stdout:1161: expect (ObjNum 507 snap 113 seq_num 507) 2023-03-29T14:07:10.432 INFO:tasks.rados.rados.0.smithi017.stdout:1162: delete oid 34 current snap is 131 2023-03-29T14:07:10.432 INFO:tasks.rados.rados.0.smithi017.stdout:1153: finishing write tid 2 to smithi017107884-39 2023-03-29T14:07:10.432 INFO:tasks.rados.rados.0.smithi017.stdout:1152: expect (ObjNum 544 snap 123 seq_num 544) 2023-03-29T14:07:10.555 INFO:tasks.rados.rados.0.smithi017.stdout:1153: finishing write tid 3 to smithi017107884-39 2023-03-29T14:07:10.555 INFO:tasks.rados.rados.0.smithi017.stdout:1153: finishing write tid 4 to smithi017107884-39 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:1153: finishing write tid 5 to smithi017107884-39 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:1153: oid 39 updating version 0 to 167 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:1153: oid 39 version 167 is already newer than 167 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:1153: oid 39 version 167 is already newer than 167 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:1153: oid 39 version 167 is already newer than 167 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 167 (ObjNum 563 snap 131 seq_num 563) dirty exists 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:1153: left oid 39 (ObjNum 563 snap 131 seq_num 563) 2023-03-29T14:07:10.556 INFO:tasks.rados.rados.0.smithi017.stdout:1156: expect (ObjNum 519 snap 116 seq_num 519) 2023-03-29T14:07:10.751 INFO:tasks.rados.rados.0.smithi017.stdout:1155: finishing write tid 1 to smithi017107884-4 2023-03-29T14:07:10.751 INFO:tasks.rados.rados.0.smithi017.stdout:1157: finishing write tid 1 to smithi017107884-27 2023-03-29T14:07:10.751 INFO:tasks.rados.rados.0.smithi017.stdout:1157: finishing write tid 2 to smithi017107884-27 2023-03-29T14:07:10.751 INFO:tasks.rados.rados.0.smithi017.stdout:1157: finishing write tid 3 to smithi017107884-27 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1155: finishing write tid 2 to smithi017107884-4 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1157: finishing write tid 4 to smithi017107884-27 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1157: finishing write tid 5 to smithi017107884-27 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1157: oid 27 updating version 0 to 120 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1157: oid 27 version 120 is already newer than 120 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1157: oid 27 version 120 is already newer than 120 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1157: oid 27 version 120 is already newer than 120 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 120 (ObjNum 565 snap 131 seq_num 565) dirty exists 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1157: left oid 27 (ObjNum 565 snap 131 seq_num 565) 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1155: finishing write tid 3 to smithi017107884-4 2023-03-29T14:07:10.752 INFO:tasks.rados.rados.0.smithi017.stdout:1155: finishing write tid 4 to smithi017107884-4 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1155: finishing write tid 5 to smithi017107884-4 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1155: oid 4 updating version 0 to 168 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1155: oid 4 updating version 168 to 172 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1155: oid 4 version 172 is already newer than 172 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1155: oid 4 version 172 is already newer than 172 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 172 (ObjNum 564 snap 131 seq_num 564) dirty exists 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1155: left oid 4 (ObjNum 564 snap 131 seq_num 564) 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1152: done (9 left) 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1153: done (8 left) 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1154: done (7 left) 2023-03-29T14:07:10.753 INFO:tasks.rados.rados.0.smithi017.stdout:1155: done (6 left) 2023-03-29T14:07:10.754 INFO:tasks.rados.rados.0.smithi017.stdout:1156: done (5 left) 2023-03-29T14:07:10.754 INFO:tasks.rados.rados.0.smithi017.stdout:1157: done (4 left) 2023-03-29T14:07:10.754 INFO:tasks.rados.rados.0.smithi017.stdout:1158: done (3 left) 2023-03-29T14:07:10.754 INFO:tasks.rados.rados.0.smithi017.stdout:1162: done (2 left) 2023-03-29T14:07:10.754 INFO:tasks.rados.rados.0.smithi017.stdout:1163: read oid 14 snap -1 2023-03-29T14:07:10.754 INFO:tasks.rados.rados.0.smithi017.stdout:1163: expect deleted 2023-03-29T14:07:10.754 INFO:tasks.rados.rados.0.smithi017.stdout:1164: snap_remove snap 112 2023-03-29T14:07:11.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~121s 2023-03-29T14:07:11.369 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:11.415 INFO:tasks.rados.rados.0.smithi017.stdout:1164: done (3 left) 2023-03-29T14:07:11.415 INFO:tasks.rados.rados.0.smithi017.stdout:1165: snap_create 2023-03-29T14:07:11.415 INFO:tasks.rados.rados.0.smithi017.stdout:1161: expect (ObjNum 507 snap 113 seq_num 507) 2023-03-29T14:07:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:11.498+0000 7f9ecf573700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:11.503+0000 7f9ecf573700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:11.566 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:11.570 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:11.570 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: finishing write tid 1 to smithi017107884-23 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: finishing write tid 2 to smithi017107884-23 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: finishing write tid 3 to smithi017107884-23 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: finishing write tid 4 to smithi017107884-23 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: finishing write tid 5 to smithi017107884-23 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: oid 23 updating version 0 to 237 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: oid 23 version 237 is already newer than 237 2023-03-29T14:07:11.590 INFO:tasks.rados.rados.0.smithi017.stdout:1160: oid 23 version 237 is already newer than 237 2023-03-29T14:07:11.591 INFO:tasks.rados.rados.0.smithi017.stdout:1160: oid 23 version 237 is already newer than 237 2023-03-29T14:07:11.591 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 237 (ObjNum 566 snap 131 seq_num 566) dirty exists 2023-03-29T14:07:11.591 INFO:tasks.rados.rados.0.smithi017.stdout:1160: left oid 23 (ObjNum 566 snap 131 seq_num 566) 2023-03-29T14:07:12.419 INFO:tasks.rados.rados.0.smithi017.stdout:1160: done (3 left) 2023-03-29T14:07:12.419 INFO:tasks.rados.rados.0.smithi017.stdout:1161: done (2 left) 2023-03-29T14:07:12.419 INFO:tasks.rados.rados.0.smithi017.stdout:1163: done (1 left) 2023-03-29T14:07:12.419 INFO:tasks.rados.rados.0.smithi017.stdout:1165: done (0 left) 2023-03-29T14:07:12.419 INFO:tasks.rados.rados.0.smithi017.stdout:1166: write oid 35 current snap is 132 2023-03-29T14:07:12.419 INFO:tasks.rados.rados.0.smithi017.stdout:1166: seq_num 568 ranges {746376=551554,2054831=649524,3315201=231175} 2023-03-29T14:07:12.425 INFO:tasks.rados.rados.0.smithi017.stdout:1166: writing smithi017107884-35 from 746376 to 1297930 tid 1 2023-03-29T14:07:12.433 INFO:tasks.rados.rados.0.smithi017.stdout:1166: writing smithi017107884-35 from 2054831 to 2704355 tid 2 2023-03-29T14:07:12.438 INFO:tasks.rados.rados.0.smithi017.stdout:1166: writing smithi017107884-35 from 3315201 to 3546376 tid 3 2023-03-29T14:07:12.438 INFO:tasks.rados.rados.0.smithi017.stdout:1167: read oid 33 snap 127 2023-03-29T14:07:12.438 INFO:tasks.rados.rados.0.smithi017.stdout:1167: expect (ObjNum 456 snap 103 seq_num 456) 2023-03-29T14:07:12.438 INFO:tasks.rados.rados.0.smithi017.stdout:1168: delete oid 26 current snap is 132 2023-03-29T14:07:12.530 INFO:tasks.rados.rados.0.smithi017.stdout:1167: expect (ObjNum 456 snap 103 seq_num 456) 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: finishing write tid 1 to smithi017107884-35 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: finishing write tid 2 to smithi017107884-35 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: finishing write tid 3 to smithi017107884-35 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: finishing write tid 4 to smithi017107884-35 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: finishing write tid 5 to smithi017107884-35 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: oid 35 updating version 0 to 85 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: oid 35 version 85 is already newer than 85 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: oid 35 version 85 is already newer than 85 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: oid 35 version 85 is already newer than 85 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 85 (ObjNum 568 snap 132 seq_num 568) dirty exists 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: left oid 35 (ObjNum 568 snap 132 seq_num 568) 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1166: done (2 left) 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1167: done (1 left) 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1168: done (0 left) 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1169: read oid 7 snap -1 2023-03-29T14:07:12.808 INFO:tasks.rados.rados.0.smithi017.stdout:1169: expect (ObjNum 521 snap 116 seq_num 521) 2023-03-29T14:07:12.809 INFO:tasks.rados.rados.0.smithi017.stdout:1170: write oid 48 current snap is 132 2023-03-29T14:07:12.809 INFO:tasks.rados.rados.0.smithi017.stdout:1170: seq_num 570 ranges {779990=702052,2055442=653193,3253694=326296} 2023-03-29T14:07:12.815 INFO:tasks.rados.rados.0.smithi017.stdout:1170: writing smithi017107884-48 from 779990 to 1482042 tid 1 2023-03-29T14:07:12.822 INFO:tasks.rados.rados.0.smithi017.stdout:1170: writing smithi017107884-48 from 2055442 to 2708635 tid 2 2023-03-29T14:07:12.827 INFO:tasks.rados.rados.0.smithi017.stdout:1170: writing smithi017107884-48 from 3253694 to 3579990 tid 3 2023-03-29T14:07:12.827 INFO:tasks.rados.rados.0.smithi017.stdout:1171: write (excl) oid 43 current snap is 132 2023-03-29T14:07:12.827 INFO:tasks.rados.rados.0.smithi017.stdout:1171: seq_num 571 ranges {796797=493654,1830277=796851,3397470=199327} 2023-03-29T14:07:12.834 INFO:tasks.rados.rados.0.smithi017.stdout:1171: writing smithi017107884-43 from 796797 to 1290451 tid 1 2023-03-29T14:07:12.842 INFO:tasks.rados.rados.0.smithi017.stdout:1171: writing smithi017107884-43 from 1830277 to 2627128 tid 2 2023-03-29T14:07:12.847 INFO:tasks.rados.rados.0.smithi017.stdout:1171: writing smithi017107884-43 from 3397470 to 3596797 tid 3 2023-03-29T14:07:12.847 INFO:tasks.rados.rados.0.smithi017.stdout:1172: snap_remove snap 131 2023-03-29T14:07:13.421 INFO:tasks.rados.rados.0.smithi017.stdout:1172: done (3 left) 2023-03-29T14:07:13.421 INFO:tasks.rados.rados.0.smithi017.stdout:1173: delete oid 33 current snap is 132 2023-03-29T14:07:13.422 INFO:tasks.rados.rados.0.smithi017.stdout:1169: expect (ObjNum 521 snap 116 seq_num 521) 2023-03-29T14:07:13.529 INFO:tasks.rados.rados.0.smithi017.stdout:1170: finishing write tid 1 to smithi017107884-48 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: finishing write tid 2 to smithi017107884-48 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1171: finishing write tid 1 to smithi017107884-43 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: finishing write tid 3 to smithi017107884-48 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: finishing write tid 4 to smithi017107884-48 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: finishing write tid 5 to smithi017107884-48 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: oid 48 updating version 0 to 177 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: oid 48 version 177 is already newer than 177 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: oid 48 version 177 is already newer than 177 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: oid 48 version 177 is already newer than 177 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 177 (ObjNum 570 snap 132 seq_num 570) dirty exists 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1170: left oid 48 (ObjNum 570 snap 132 seq_num 570) 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1171: finishing write tid 2 to smithi017107884-43 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1171: finishing write tid 3 to smithi017107884-43 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1171: finishing write tid 4 to smithi017107884-43 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1171: finishing write tid 5 to smithi017107884-43 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1171: oid 43 updating version 0 to 101 2023-03-29T14:07:13.530 INFO:tasks.rados.rados.0.smithi017.stdout:1171: oid 43 updating version 101 to 105 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1171: oid 43 version 105 is already newer than 105 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1171: oid 43 version 105 is already newer than 105 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 105 (ObjNum 571 snap 132 seq_num 571) dirty exists 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1171: left oid 43 (ObjNum 571 snap 132 seq_num 571) 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1169: done (3 left) 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1170: done (2 left) 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1171: done (1 left) 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1173: done (0 left) 2023-03-29T14:07:13.531 INFO:tasks.rados.rados.0.smithi017.stdout:1174: snap_create 2023-03-29T14:07:13.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:13.701+0000 7f0e273b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:13.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:13.705+0000 7f0e273b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:13.766 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:13.769 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:13.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:14.208 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:14.208 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:14.208 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:14.336+0000 7fe93d987700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:14.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:14.343+0000 7fe93d987700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:14.428 INFO:tasks.rados.rados.0.smithi017.stdout:1174: done (0 left) 2023-03-29T14:07:14.428 INFO:tasks.rados.rados.0.smithi017.stdout:1175: write (excl) oid 9 current snap is 133 2023-03-29T14:07:14.428 INFO:tasks.rados.rados.0.smithi017.stdout:1175: seq_num 573 ranges {430411=644152,1830888=400520,2935963=402981} 2023-03-29T14:07:14.434 INFO:tasks.rados.rados.0.smithi017.stdout:1175: writing smithi017107884-9 from 430411 to 1074563 tid 1 2023-03-29T14:07:14.440 INFO:tasks.rados.rados.0.smithi017.stdout:1175: writing smithi017107884-9 from 1830888 to 2231408 tid 2 2023-03-29T14:07:14.446 INFO:tasks.rados.rados.0.smithi017.stdout:1175: writing smithi017107884-9 from 2935963 to 3338944 tid 3 2023-03-29T14:07:14.446 INFO:tasks.rados.rados.0.smithi017.stdout:1176: delete oid 39 current snap is 133 2023-03-29T14:07:14.485 INFO:tasks.rados.rados.0.smithi017.stdout:1175: finishing write tid 1 to smithi017107884-9 2023-03-29T14:07:14.554 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:14.556 INFO:tasks.rados.rados.0.smithi017.stdout:1176: done (1 left) 2023-03-29T14:07:14.556 INFO:tasks.rados.rados.0.smithi017.stdout:1177: read oid 19 snap -1 2023-03-29T14:07:14.556 INFO:tasks.rados.rados.0.smithi017.stdout:1177: expect deleted 2023-03-29T14:07:14.556 INFO:tasks.rados.rados.0.smithi017.stdout:1178: snap_remove snap 127 2023-03-29T14:07:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:14.569 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2023-03-29T14:07:14.425787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":207072888,"num_objects":636,"num_object_clones":540,"num_object_copies":1908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":2670,"num_read_kb":1505808,"num_write":3682,"num_write_kb":993748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1928,"ondisk_log_size":1928,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1271216,"kb_used_data":1206480,"kb_used_omap":0,"kb_used_meta":64704,"kb_avail":376216144,"statfs":{"total":386547056640,"available":385245331456,"internally_reserved":0,"allocated":1235435520,"data_stored":1491234590,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":66256896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6084134,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":63,"num_read_kb":47368,"num_write":100,"num_write_kb":31024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015508"},"pg_stats":[{"pgid":"3.f","version":"277'88","reported_seq":210,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:09.417367+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:09.417367+0000","last_peered":"2023-03-29T14:07:09.417367+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:09.417367+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12235892,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":34315,"num_write":98,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":301,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:09.457197+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:09.457197+0000","last_peered":"2023-03-29T14:07:09.457197+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:09.457197+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"283'172","reported_seq":613,"reported_epoch":284,"state":"active+clean","last_fresh":"2023-03-29T14:07:09.688904+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:09.688904+0000","last_peered":"2023-03-29T14:07:09.688904+0000","last_clean":"2023-03-29T14:07:09.688904+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:09.688904+0000","last_undegraded":"2023-03-29T14:07:09.688904+0000","last_fullsized":"2023-03-29T14:07:09.688904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488624,"num_objects":58,"num_object_clones":48,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":236,"num_read_kb":151255,"num_write":328,"num_write_kb":96672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"282'162","reported_seq":607,"reported_epoch":284,"state":"active+clean","last_fresh":"2023-03-29T14:07:10.446632+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:10.446632+0000","last_peered":"2023-03-29T14:07:10.446632+0000","last_clean":"2023-03-29T14:07:10.446632+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:10.446632+0000","last_undegraded":"2023-03-29T14:07:10.446632+0000","last_fullsized":"2023-03-29T14:07:10.446632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18017698,"num_objects":66,"num_object_clones":58,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":258,"num_read_kb":158286,"num_write":318,"num_write_kb":89706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"282'175","reported_seq":604,"reported_epoch":284,"state":"active+clean","last_fresh":"2023-03-29T14:07:10.506704+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:10.506704+0000","last_peered":"2023-03-29T14:07:10.506704+0000","last_clean":"2023-03-29T14:07:10.506704+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:10.506704+0000","last_undegraded":"2023-03-29T14:07:10.506704+0000","last_fullsized":"2023-03-29T14:07:10.506704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12303120,"num_objects":66,"num_object_clones":58,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":218,"num_read_kb":120996,"num_write":354,"num_write_kb":102714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":237,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.420506+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:09.420506+0000","last_peered":"2023-03-29T14:07:09.420506+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:09.420506+0000","last_undegraded":"2023-03-29T14:07:09.420506+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"260'104","reported_seq":459,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:09.423535+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:09.423535+0000","last_peered":"2023-03-29T14:07:09.423535+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:09.423535+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748192,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":127,"num_read_kb":61239,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'69","reported_seq":387,"reported_epoch":284,"state":"active+clean","last_fresh":"2023-03-29T14:07:09.426428+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:09.426428+0000","last_peered":"2023-03-29T14:07:09.426428+0000","last_clean":"2023-03-29T14:07:09.426428+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:09.426428+0000","last_undegraded":"2023-03-29T14:07:09.426428+0000","last_fullsized":"2023-03-29T14:07:09.426428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":95,"num_read_kb":52821,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.451980+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:09.451980+0000","last_peered":"2023-03-29T14:07:09.451980+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:09.451980+0000","last_undegraded":"2023-03-29T14:07:09.451980+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'65","reported_seq":144,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:08.392538+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:08.392538+0000","last_peered":"2023-03-29T14:07:08.392538+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:08.392538+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":240,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.428890+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:09.428890+0000","last_peered":"2023-03-29T14:07:09.428890+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:09.428890+0000","last_undegraded":"2023-03-29T14:07:09.428890+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":410,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:08.435278+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:08.435278+0000","last_peered":"2023-03-29T14:07:08.435278+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:08.435278+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":85,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.460689+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:09.460689+0000","last_peered":"2023-03-29T14:07:09.460689+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:09.460689+0000","last_undegraded":"2023-03-29T14:07:09.460689+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"279'100","reported_seq":440,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:09.470785+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:09.470785+0000","last_peered":"2023-03-29T14:07:09.470785+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:09.470785+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12941786,"num_objects":36,"num_object_clones":30,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":121,"num_read_kb":58664,"num_write":198,"num_write_kb":58720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":240,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.422837+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:09.422837+0000","last_peered":"2023-03-29T14:07:09.422837+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:09.422837+0000","last_undegraded":"2023-03-29T14:07:09.422837+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":449,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:09.476447+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:09.476447+0000","last_peered":"2023-03-29T14:07:09.476447+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:09.476447+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"283'120","reported_seq":556,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:09.684601+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:09.684601+0000","last_peered":"2023-03-29T14:07:09.684601+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:09.684601+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22891926,"num_objects":40,"num_object_clones":32,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":179139,"num_write":242,"num_write_kb":69906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":240,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.430387+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:09.430387+0000","last_peered":"2023-03-29T14:07:09.430387+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:09.430387+0000","last_undegraded":"2023-03-29T14:07:09.430387+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"284'237","reported_seq":778,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:10.562497+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:10.562497+0000","last_peered":"2023-03-29T14:07:10.562497+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:10.562497+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36842132,"num_objects":80,"num_object_clones":66,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":389,"num_read_kb":238805,"num_write":446,"num_write_kb":128634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":80}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":224,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.478478+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:09.478478+0000","last_peered":"2023-03-29T14:07:09.478478+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:09.478478+0000","last_undegraded":"2023-03-29T14:07:09.478478+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"282'80","reported_seq":403,"reported_epoch":284,"state":"active+clean","last_fresh":"2023-03-29T14:07:09.446307+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:09.446307+0000","last_peered":"2023-03-29T14:07:09.446307+0000","last_clean":"2023-03-29T14:07:09.446307+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:09.446307+0000","last_undegraded":"2023-03-29T14:07:09.446307+0000","last_fullsized":"2023-03-29T14:07:09.446307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823840,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":121,"num_read_kb":70177,"num_write":166,"num_write_kb":47670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"282'184","reported_seq":615,"reported_epoch":284,"state":"active+clean","last_fresh":"2023-03-29T14:07:09.539458+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:09.539458+0000","last_peered":"2023-03-29T14:07:09.539458+0000","last_clean":"2023-03-29T14:07:09.539458+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:09.539458+0000","last_undegraded":"2023-03-29T14:07:09.539458+0000","last_fullsized":"2023-03-29T14:07:09.539458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":229,"num_read_kb":133906,"num_write":344,"num_write_kb":100514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.480517+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:09.480517+0000","last_peered":"2023-03-29T14:07:09.480517+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:09.480517+0000","last_undegraded":"2023-03-29T14:07:09.480517+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":240,"reported_epoch":284,"state":"active+undersized","last_fresh":"2023-03-29T14:07:09.419218+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:09.419218+0000","last_peered":"2023-03-29T14:07:09.419218+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:09.419218+0000","last_undegraded":"2023-03-29T14:07:09.419218+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"273'159","reported_seq":575,"reported_epoch":284,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:08.413488+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:08.413488+0000","last_peered":"2023-03-29T14:07:08.413488+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:08.413488+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19765560,"num_objects":56,"num_object_clones":48,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":204,"num_read_kb":119530,"num_write":310,"num_write_kb":93760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":56}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":205892152,"num_objects":632,"num_object_clones":540,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":2528,"num_read_kb":1505688,"num_write":3408,"num_write_kb":989960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1831,"ondisk_log_size":1831,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":351928,"kb_used_data":336368,"kb_used_omap":0,"kb_used_meta":15552,"kb_avail":94019912,"statfs":{"total":96636764160,"available":96276389888,"internally_reserved":0,"allocated":344440832,"data_stored":414017841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15925248},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":403028,"kb_used_data":382156,"kb_used_omap":0,"kb_used_meta":20864,"kb_avail":93968812,"statfs":{"total":96636764160,"available":96224063488,"internally_reserved":0,"allocated":391327744,"data_stored":475827100,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21364736},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":360316,"kb_used_data":344116,"kb_used_omap":0,"kb_used_meta":16192,"kb_avail":94011524,"statfs":{"total":96636764160,"available":96267800576,"internally_reserved":0,"allocated":352374784,"data_stored":428157424,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16580608},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:14.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.9 3.5 3.0 3.2 3.1 3.6 2023-03-29T14:07:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:14.704+0000 7f39f4b9a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:14.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:14.719+0000 7f39f4b9a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:14.929 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.5,3.6] on osd.0 to force-recovery; instructing pg(s) [3.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.2,3.9,3.f] on osd.3 to force-recovery; 2023-03-29T14:07:14.929 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:07:15.434 INFO:tasks.rados.rados.0.smithi017.stdout:1178: done (2 left) 2023-03-29T14:07:15.434 INFO:tasks.rados.rados.0.smithi017.stdout:1179: snap_create 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: finishing write tid 2 to smithi017107884-9 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: finishing write tid 3 to smithi017107884-9 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: finishing write tid 4 to smithi017107884-9 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: finishing write tid 5 to smithi017107884-9 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: oid 9 updating version 0 to 167 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: oid 9 version 167 is already newer than 167 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: oid 9 version 167 is already newer than 167 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:1175: oid 9 version 167 is already newer than 163 2023-03-29T14:07:15.435 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 167 (ObjNum 573 snap 133 seq_num 573) dirty exists 2023-03-29T14:07:15.436 INFO:tasks.rados.rados.0.smithi017.stdout:1175: left oid 9 (ObjNum 573 snap 133 seq_num 573) 2023-03-29T14:07:15.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:15.899 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:15.898+0000 7fad482fa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:15.903 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:15.903+0000 7fad482fa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:15.966 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:15.969 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:15.969 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:16.434 INFO:tasks.rados.rados.0.smithi017.stdout:1175: done (2 left) 2023-03-29T14:07:16.434 INFO:tasks.rados.rados.0.smithi017.stdout:1177: done (1 left) 2023-03-29T14:07:16.434 INFO:tasks.rados.rados.0.smithi017.stdout:1179: done (0 left) 2023-03-29T14:07:16.434 INFO:tasks.rados.rados.0.smithi017.stdout:1180: snap_create 2023-03-29T14:07:16.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~127s 2023-03-29T14:07:17.435 INFO:tasks.rados.rados.0.smithi017.stdout:1180: done (0 left) 2023-03-29T14:07:17.435 INFO:tasks.rados.rados.0.smithi017.stdout:1181: write oid 23 current snap is 135 2023-03-29T14:07:17.435 INFO:tasks.rados.rados.0.smithi017.stdout:1181: seq_num 575 ranges {464025=794650,1947852=404189,2990809=425525} 2023-03-29T14:07:17.442 INFO:tasks.rados.rados.0.smithi017.stdout:1181: writing smithi017107884-23 from 464025 to 1258675 tid 1 2023-03-29T14:07:17.449 INFO:tasks.rados.rados.0.smithi017.stdout:1181: writing smithi017107884-23 from 1947852 to 2352041 tid 2 2023-03-29T14:07:17.454 INFO:tasks.rados.rados.0.smithi017.stdout:1181: writing smithi017107884-23 from 2990809 to 3416334 tid 3 2023-03-29T14:07:17.454 INFO:tasks.rados.rados.0.smithi017.stdout:1182: snap_create 2023-03-29T14:07:17.562 INFO:tasks.rados.rados.0.smithi017.stdout:1181: finishing write tid 1 to smithi017107884-23 2023-03-29T14:07:17.564 INFO:tasks.rados.rados.0.smithi017.stdout:1181: finishing write tid 2 to smithi017107884-23 2023-03-29T14:07:17.566 INFO:tasks.rados.rados.0.smithi017.stdout:1181: finishing write tid 3 to smithi017107884-23 2023-03-29T14:07:17.567 INFO:tasks.rados.rados.0.smithi017.stdout:1181: finishing write tid 4 to smithi017107884-23 2023-03-29T14:07:17.570 INFO:tasks.rados.rados.0.smithi017.stdout:1181: finishing write tid 5 to smithi017107884-23 2023-03-29T14:07:17.570 INFO:tasks.rados.rados.0.smithi017.stdout:1181: oid 23 updating version 0 to 242 2023-03-29T14:07:17.570 INFO:tasks.rados.rados.0.smithi017.stdout:1181: oid 23 version 242 is already newer than 242 2023-03-29T14:07:17.571 INFO:tasks.rados.rados.0.smithi017.stdout:1181: oid 23 version 242 is already newer than 242 2023-03-29T14:07:17.571 INFO:tasks.rados.rados.0.smithi017.stdout:1181: oid 23 version 242 is already newer than 242 2023-03-29T14:07:17.571 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 242 (ObjNum 575 snap 135 seq_num 575) dirty exists 2023-03-29T14:07:17.571 INFO:tasks.rados.rados.0.smithi017.stdout:1181: left oid 23 (ObjNum 575 snap 135 seq_num 575) 2023-03-29T14:07:17.971 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:18.100 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:18.098+0000 7f21781eb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:18.103 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:18.102+0000 7f21781eb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:18.175 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:18.178 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:18.179 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:18.232 INFO:tasks.rados.rados.0.smithi017.stdout:1181: done (1 left) 2023-03-29T14:07:18.232 INFO:tasks.rados.rados.0.smithi017.stdout:1182: done (0 left) 2023-03-29T14:07:18.232 INFO:tasks.rados.rados.0.smithi017.stdout:1183: snap_remove snap 134 2023-03-29T14:07:19.235 INFO:tasks.rados.rados.0.smithi017.stdout:1183: done (0 left) 2023-03-29T14:07:19.236 INFO:tasks.rados.rados.0.smithi017.stdout:1184: read oid 15 snap 133 2023-03-29T14:07:19.236 INFO:tasks.rados.rados.0.smithi017.stdout:1184: expect (ObjNum 559 snap 131 seq_num 559) 2023-03-29T14:07:19.236 INFO:tasks.rados.rados.0.smithi017.stdout:1185: read oid 4 snap 133 2023-03-29T14:07:19.236 INFO:tasks.rados.rados.0.smithi017.stdout:1185: expect (ObjNum 564 snap 131 seq_num 564) 2023-03-29T14:07:19.236 INFO:tasks.rados.rados.0.smithi017.stdout:1186: write oid 39 current snap is 136 2023-03-29T14:07:19.236 INFO:tasks.rados.rados.0.smithi017.stdout:1186: seq_num 576 ranges {480832=469899,1606334=664200,3018232=662600} 2023-03-29T14:07:19.240 INFO:tasks.rados.rados.0.smithi017.stdout:1186: writing smithi017107884-39 from 480832 to 950731 tid 1 2023-03-29T14:07:19.248 INFO:tasks.rados.rados.0.smithi017.stdout:1186: writing smithi017107884-39 from 1606334 to 2270534 tid 2 2023-03-29T14:07:19.255 INFO:tasks.rados.rados.0.smithi017.stdout:1186: writing smithi017107884-39 from 3018232 to 3680832 tid 3 2023-03-29T14:07:19.256 INFO:tasks.rados.rados.0.smithi017.stdout:1187: snap_create 2023-03-29T14:07:19.282 INFO:tasks.rados.rados.0.smithi017.stdout:1185: expect (ObjNum 564 snap 131 seq_num 564) 2023-03-29T14:07:19.580 INFO:tasks.rados.rados.0.smithi017.stdout:1186: finishing write tid 1 to smithi017107884-39 2023-03-29T14:07:19.580 INFO:tasks.rados.rados.0.smithi017.stdout:1186: finishing write tid 2 to smithi017107884-39 2023-03-29T14:07:19.580 INFO:tasks.rados.rados.0.smithi017.stdout:1184: expect (ObjNum 559 snap 131 seq_num 559) 2023-03-29T14:07:19.889 INFO:tasks.rados.rados.0.smithi017.stdout:1186: finishing write tid 3 to smithi017107884-39 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:1186: finishing write tid 4 to smithi017107884-39 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:1186: finishing write tid 5 to smithi017107884-39 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:1186: oid 39 updating version 0 to 183 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:1186: oid 39 version 183 is already newer than 183 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:1186: oid 39 version 183 is already newer than 183 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:1186: oid 39 version 183 is already newer than 183 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 183 (ObjNum 576 snap 136 seq_num 576) dirty exists 2023-03-29T14:07:19.890 INFO:tasks.rados.rados.0.smithi017.stdout:1186: left oid 39 (ObjNum 576 snap 136 seq_num 576) 2023-03-29T14:07:19.941 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:19.941 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:19.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:20.069+0000 7f26ee4cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:20.076 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:20.075+0000 7f26ee4cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:20.180 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:1184: done (3 left) 2023-03-29T14:07:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:1185: done (2 left) 2023-03-29T14:07:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:1186: done (1 left) 2023-03-29T14:07:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:1187: done (0 left) 2023-03-29T14:07:20.237 INFO:tasks.rados.rados.0.smithi017.stdout:1188: snap_remove snap 135 2023-03-29T14:07:20.283 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:20.296 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2023-03-29T14:07:20.235537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189794764,"num_objects":650,"num_object_clones":554,"num_object_copies":1950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2694,"num_read_kb":1519251,"num_write":3722,"num_write_kb":1005580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1955,"ondisk_log_size":1955,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1288356,"kb_used_data":1223172,"kb_used_omap":0,"kb_used_meta":65152,"kb_avail":376199004,"statfs":{"total":386547056640,"available":385227780096,"internally_reserved":0,"allocated":1252528128,"data_stored":1519624869,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":66715648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11093016,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":13,"num_read_kb":5146,"num_write":20,"num_write_kb":6118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.802826"},"pg_stats":[{"pgid":"3.f","version":"277'88","reported_seq":214,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:14.449503+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:14.449503+0000","last_peered":"2023-03-29T14:07:14.449503+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:14.449503+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12235892,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":34315,"num_write":98,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":305,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:14.576595+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:14.576595+0000","last_peered":"2023-03-29T14:07:14.576595+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:14.576595+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"288'179","reported_seq":628,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-03-29T14:07:14.478075+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:14.478075+0000","last_peered":"2023-03-29T14:07:14.478075+0000","last_clean":"2023-03-29T14:07:14.478075+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:14.478075+0000","last_undegraded":"2023-03-29T14:07:14.478075+0000","last_fullsized":"2023-03-29T14:07:14.478075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20236222,"num_objects":62,"num_object_clones":52,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":239,"num_read_kb":151256,"num_write":338,"num_write_kb":99958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"288'167","reported_seq":620,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-03-29T14:07:14.644025+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:14.644025+0000","last_peered":"2023-03-29T14:07:14.644025+0000","last_clean":"2023-03-29T14:07:14.644025+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:14.644025+0000","last_undegraded":"2023-03-29T14:07:14.644025+0000","last_fullsized":"2023-03-29T14:07:14.644025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521908,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":261,"num_read_kb":158287,"num_write":328,"num_write_kb":92538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"284'176","reported_seq":608,"reported_epoch":289,"state":"active+clean","last_fresh":"2023-03-29T14:07:14.536155+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:14.536155+0000","last_peered":"2023-03-29T14:07:14.536155+0000","last_clean":"2023-03-29T14:07:14.536155+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:14.536155+0000","last_undegraded":"2023-03-29T14:07:14.536155+0000","last_fullsized":"2023-03-29T14:07:14.536155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12303120,"num_objects":66,"num_object_clones":58,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":218,"num_read_kb":120996,"num_write":354,"num_write_kb":102714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":241,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.433948+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:14.433948+0000","last_peered":"2023-03-29T14:07:14.433948+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:14.433948+0000","last_undegraded":"2023-03-29T14:07:14.433948+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"260'104","reported_seq":463,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:14.446286+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:14.446286+0000","last_peered":"2023-03-29T14:07:14.446286+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:14.446286+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748192,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":127,"num_read_kb":61239,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'69","reported_seq":394,"reported_epoch":290,"state":"active+clean","last_fresh":"2023-03-29T14:07:14.444928+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:14.444928+0000","last_peered":"2023-03-29T14:07:14.444928+0000","last_clean":"2023-03-29T14:07:14.444928+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:14.444928+0000","last_undegraded":"2023-03-29T14:07:14.444928+0000","last_fullsized":"2023-03-29T14:07:14.444928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":57965,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":228,"reported_epoch":289,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.555195+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:14.555195+0000","last_peered":"2023-03-29T14:07:14.555195+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:14.555195+0000","last_undegraded":"2023-03-29T14:07:14.555195+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'65","reported_seq":148,"reported_epoch":288,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:13.457858+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:13.457858+0000","last_peered":"2023-03-29T14:07:13.457858+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:13.457858+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":244,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.447011+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:14.447011+0000","last_peered":"2023-03-29T14:07:14.447011+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:14.447011+0000","last_undegraded":"2023-03-29T14:07:14.447011+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":414,"reported_epoch":288,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:13.473586+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:13.473586+0000","last_peered":"2023-03-29T14:07:13.473586+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:13.473586+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":289,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.564634+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:14.564634+0000","last_peered":"2023-03-29T14:07:14.564634+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:14.564634+0000","last_undegraded":"2023-03-29T14:07:14.564634+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"286'105","reported_seq":453,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:14.562157+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:14.562157+0000","last_peered":"2023-03-29T14:07:14.562157+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:14.562157+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13815750,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":58665,"num_write":208,"num_write_kb":61634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":244,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.440311+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:14.440311+0000","last_peered":"2023-03-29T14:07:14.440311+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:14.440311+0000","last_undegraded":"2023-03-29T14:07:14.440311+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":453,"reported_epoch":289,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:14.561234+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:14.561234+0000","last_peered":"2023-03-29T14:07:14.561234+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:14.561234+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"286'122","reported_seq":562,"reported_epoch":290,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:14.448208+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:14.448208+0000","last_peered":"2023-03-29T14:07:14.448208+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:14.448208+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17547102,"num_objects":42,"num_object_clones":34,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":259,"num_read_kb":179139,"num_write":242,"num_write_kb":69906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":42}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.439578+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:14.439578+0000","last_peered":"2023-03-29T14:07:14.439578+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:14.439578+0000","last_undegraded":"2023-03-29T14:07:14.439578+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"284'237","reported_seq":782,"reported_epoch":288,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:13.467467+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:13.467467+0000","last_peered":"2023-03-29T14:07:13.467467+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:13.467467+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36842132,"num_objects":80,"num_object_clones":66,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":389,"num_read_kb":238805,"num_write":446,"num_write_kb":128634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":80}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":228,"reported_epoch":289,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.568585+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:14.568585+0000","last_peered":"2023-03-29T14:07:14.568585+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:14.568585+0000","last_undegraded":"2023-03-29T14:07:14.568585+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"286'85","reported_seq":416,"reported_epoch":289,"state":"active+clean","last_fresh":"2023-03-29T14:07:14.542571+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:14.542571+0000","last_peered":"2023-03-29T14:07:14.542571+0000","last_clean":"2023-03-29T14:07:14.542571+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:14.542571+0000","last_undegraded":"2023-03-29T14:07:14.542571+0000","last_fullsized":"2023-03-29T14:07:14.542571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7092752,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":122,"num_read_kb":70178,"num_write":176,"num_write_kb":50470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"282'184","reported_seq":619,"reported_epoch":289,"state":"active+clean","last_fresh":"2023-03-29T14:07:14.579951+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:14.579951+0000","last_peered":"2023-03-29T14:07:14.579951+0000","last_clean":"2023-03-29T14:07:14.579951+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:14.579951+0000","last_undegraded":"2023-03-29T14:07:14.579951+0000","last_fullsized":"2023-03-29T14:07:14.579951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":229,"num_read_kb":133906,"num_write":344,"num_write_kb":100514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":228,"reported_epoch":289,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.537810+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:14.537810+0000","last_peered":"2023-03-29T14:07:14.537810+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:14.537810+0000","last_undegraded":"2023-03-29T14:07:14.537810+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":244,"reported_epoch":290,"state":"active+undersized","last_fresh":"2023-03-29T14:07:14.445574+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:14.445574+0000","last_peered":"2023-03-29T14:07:14.445574+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:14.445574+0000","last_undegraded":"2023-03-29T14:07:14.445574+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"287'161","reported_seq":584,"reported_epoch":288,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:13.490680+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:13.490680+0000","last_peered":"2023-03-29T14:07:13.490680+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:13.490680+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12437576,"num_objects":58,"num_object_clones":50,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":211,"num_read_kb":127825,"num_write":310,"num_write_kb":93760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":58}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":188614028,"num_objects":646,"num_object_clones":554,"num_object_copies":1938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2552,"num_read_kb":1519131,"num_write":3448,"num_write_kb":1001792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1858,"ondisk_log_size":1858,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":356568,"kb_used_data":340880,"kb_used_omap":0,"kb_used_meta":15680,"kb_avail":94015272,"statfs":{"total":96636764160,"available":96271638528,"internally_reserved":0,"allocated":349061120,"data_stored":421951572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16056320},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":409408,"kb_used_data":388344,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":93962432,"statfs":{"total":96636764160,"available":96217530368,"internally_reserved":0,"allocated":397664256,"data_stored":486488519,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":366436,"kb_used_data":350108,"kb_used_omap":0,"kb_used_meta":16320,"kb_avail":94005404,"statfs":{"total":96636764160,"available":96261533696,"internally_reserved":0,"allocated":358510592,"data_stored":437952553,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16711680},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:20.299 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.9 3.5 1.0 3.1 2023-03-29T14:07:20.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:20.310+0000 7fc342a79700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:20.316 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:20.315+0000 7fc342a79700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:20.392 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:20.396 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:20.396 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:20.429+0000 7fb509e8e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:20.442+0000 7fb509e8e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:20.649 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.5] on osd.0 to force-recovery; instructing pg(s) [1.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.9,3.f] on osd.3 to force-recovery; 2023-03-29T14:07:20.649 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:07:21.238 INFO:tasks.rados.rados.0.smithi017.stdout:1188: done (0 left) 2023-03-29T14:07:21.238 INFO:tasks.rados.rados.0.smithi017.stdout:1189: write (excl) oid 32 current snap is 137 2023-03-29T14:07:21.238 INFO:tasks.rados.rados.0.smithi017.stdout:1189: seq_num 577 ranges {497639=545148,1548463=407858,2529302=564422,3528143=169496} 2023-03-29T14:07:21.244 INFO:tasks.rados.rados.0.smithi017.stdout:1189: writing smithi017107884-32 from 497639 to 1042787 tid 1 2023-03-29T14:07:21.248 INFO:tasks.rados.rados.0.smithi017.stdout:1189: writing smithi017107884-32 from 1548463 to 1956321 tid 2 2023-03-29T14:07:21.255 INFO:tasks.rados.rados.0.smithi017.stdout:1189: writing smithi017107884-32 from 2529302 to 3093724 tid 3 2023-03-29T14:07:21.258 INFO:tasks.rados.rados.0.smithi017.stdout:1189: writing smithi017107884-32 from 3528143 to 3697639 tid 4 2023-03-29T14:07:21.258 INFO:tasks.rados.rados.0.smithi017.stdout:1190: snap_remove snap 133 2023-03-29T14:07:22.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~132s 2023-03-29T14:07:22.244 INFO:tasks.rados.rados.0.smithi017.stdout:1190: done (1 left) 2023-03-29T14:07:22.244 INFO:tasks.rados.rados.0.smithi017.stdout:1191: snap_remove snap 128 2023-03-29T14:07:22.397 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:22.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:22.524+0000 7f4fd8478700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:22.530 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:22.529+0000 7f4fd8478700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:22.593 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:22.596 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:22.597 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:23.245 INFO:tasks.rados.rados.0.smithi017.stdout:1191: done (1 left) 2023-03-29T14:07:23.245 INFO:tasks.rados.rados.0.smithi017.stdout:1192: delete oid 37 current snap is 137 2023-03-29T14:07:23.245 INFO:tasks.rados.rados.0.smithi017.stdout:1189: finishing write tid 1 to smithi017107884-32 2023-03-29T14:07:23.245 INFO:tasks.rados.rados.0.smithi017.stdout:1189: finishing write tid 2 to smithi017107884-32 2023-03-29T14:07:23.245 INFO:tasks.rados.rados.0.smithi017.stdout:1189: finishing write tid 3 to smithi017107884-32 2023-03-29T14:07:23.245 INFO:tasks.rados.rados.0.smithi017.stdout:1189: finishing write tid 4 to smithi017107884-32 2023-03-29T14:07:23.245 INFO:tasks.rados.rados.0.smithi017.stdout:1189: finishing write tid 5 to smithi017107884-32 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:1189: finishing write tid 6 to smithi017107884-32 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:1189: oid 32 updating version 0 to 243 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:1189: oid 32 updating version 243 to 248 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:1189: oid 32 version 248 is already newer than 248 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:1189: oid 32 version 248 is already newer than 248 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:1189: oid 32 version 248 is already newer than 248 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 248 (ObjNum 577 snap 137 seq_num 577) dirty exists 2023-03-29T14:07:23.246 INFO:tasks.rados.rados.0.smithi017.stdout:1189: left oid 32 (ObjNum 577 snap 137 seq_num 577) 2023-03-29T14:07:23.364 INFO:tasks.rados.rados.0.smithi017.stdout:1189: done (1 left) 2023-03-29T14:07:23.364 INFO:tasks.rados.rados.0.smithi017.stdout:1192: done (0 left) 2023-03-29T14:07:23.364 INFO:tasks.rados.rados.0.smithi017.stdout:1193: write oid 25 current snap is 137 2023-03-29T14:07:23.364 INFO:tasks.rados.rados.0.smithi017.stdout:1193: seq_num 579 ranges {531253=411999,1381780=411527,2300501=586966,3376974=354279} 2023-03-29T14:07:23.369 INFO:tasks.rados.rados.0.smithi017.stdout:1193: writing smithi017107884-25 from 531253 to 943252 tid 1 2023-03-29T14:07:23.374 INFO:tasks.rados.rados.0.smithi017.stdout:1193: writing smithi017107884-25 from 1381780 to 1793307 tid 2 2023-03-29T14:07:23.380 INFO:tasks.rados.rados.0.smithi017.stdout:1193: writing smithi017107884-25 from 2300501 to 2887467 tid 3 2023-03-29T14:07:23.386 INFO:tasks.rados.rados.0.smithi017.stdout:1193: writing smithi017107884-25 from 3376974 to 3731253 tid 4 2023-03-29T14:07:23.386 INFO:tasks.rados.rados.0.smithi017.stdout:1194: read oid 48 snap -1 2023-03-29T14:07:23.386 INFO:tasks.rados.rados.0.smithi017.stdout:1194: expect (ObjNum 570 snap 132 seq_num 570) 2023-03-29T14:07:23.386 INFO:tasks.rados.rados.0.smithi017.stdout:1195: delete oid 7 current snap is 137 2023-03-29T14:07:23.404 INFO:tasks.rados.rados.0.smithi017.stdout:1193: finishing write tid 1 to smithi017107884-25 2023-03-29T14:07:23.411 INFO:tasks.rados.rados.0.smithi017.stdout:1193: finishing write tid 2 to smithi017107884-25 2023-03-29T14:07:23.430 INFO:tasks.rados.rados.0.smithi017.stdout:1193: finishing write tid 3 to smithi017107884-25 2023-03-29T14:07:23.434 INFO:tasks.rados.rados.0.smithi017.stdout:1193: finishing write tid 4 to smithi017107884-25 2023-03-29T14:07:23.438 INFO:tasks.rados.rados.0.smithi017.stdout:1194: expect (ObjNum 570 snap 132 seq_num 570) 2023-03-29T14:07:23.708 INFO:tasks.rados.rados.0.smithi017.stdout:1193: finishing write tid 5 to smithi017107884-25 2023-03-29T14:07:23.708 INFO:tasks.rados.rados.0.smithi017.stdout:1193: finishing write tid 6 to smithi017107884-25 2023-03-29T14:07:23.708 INFO:tasks.rados.rados.0.smithi017.stdout:1193: oid 25 updating version 0 to 173 2023-03-29T14:07:23.708 INFO:tasks.rados.rados.0.smithi017.stdout:1193: oid 25 version 173 is already newer than 173 2023-03-29T14:07:23.708 INFO:tasks.rados.rados.0.smithi017.stdout:1193: oid 25 version 173 is already newer than 173 2023-03-29T14:07:23.708 INFO:tasks.rados.rados.0.smithi017.stdout:1193: oid 25 version 173 is already newer than 171 2023-03-29T14:07:23.709 INFO:tasks.rados.rados.0.smithi017.stdout:1193: oid 25 version 173 is already newer than 173 2023-03-29T14:07:23.709 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 173 (ObjNum 579 snap 137 seq_num 579) dirty exists 2023-03-29T14:07:23.709 INFO:tasks.rados.rados.0.smithi017.stdout:1193: left oid 25 (ObjNum 579 snap 137 seq_num 579) 2023-03-29T14:07:23.709 INFO:tasks.rados.rados.0.smithi017.stdout:1193: done (2 left) 2023-03-29T14:07:23.709 INFO:tasks.rados.rados.0.smithi017.stdout:1194: done (1 left) 2023-03-29T14:07:23.709 INFO:tasks.rados.rados.0.smithi017.stdout:1195: done (0 left) 2023-03-29T14:07:23.710 INFO:tasks.rados.rados.0.smithi017.stdout:1196: write (excl) oid 6 current snap is 137 2023-03-29T14:07:23.710 INFO:tasks.rados.rados.0.smithi017.stdout:1196: seq_num 581 ranges {564867=562497,1898744=415196,2755347=725863} 2023-03-29T14:07:23.716 INFO:tasks.rados.rados.0.smithi017.stdout:1196: writing smithi017107884-6 from 564867 to 1127364 tid 1 2023-03-29T14:07:23.724 INFO:tasks.rados.rados.0.smithi017.stdout:1196: writing smithi017107884-6 from 1898744 to 2313940 tid 2 2023-03-29T14:07:23.733 INFO:tasks.rados.rados.0.smithi017.stdout:1196: writing smithi017107884-6 from 2755347 to 3481210 tid 3 2023-03-29T14:07:23.733 INFO:tasks.rados.rados.0.smithi017.stdout:1197: snap_create 2023-03-29T14:07:23.769 INFO:tasks.rados.rados.0.smithi017.stdout:1196: finishing write tid 1 to smithi017107884-6 2023-03-29T14:07:23.783 INFO:tasks.rados.rados.0.smithi017.stdout:1196: finishing write tid 2 to smithi017107884-6 2023-03-29T14:07:23.825 INFO:tasks.rados.rados.0.smithi017.stdout:1196: finishing write tid 3 to smithi017107884-6 2023-03-29T14:07:23.825 INFO:tasks.rados.rados.0.smithi017.stdout:1196: finishing write tid 4 to smithi017107884-6 2023-03-29T14:07:23.826 INFO:tasks.rados.rados.0.smithi017.stdout:1196: finishing write tid 5 to smithi017107884-6 2023-03-29T14:07:23.827 INFO:tasks.rados.rados.0.smithi017.stdout:1196: oid 6 updating version 0 to 127 2023-03-29T14:07:23.827 INFO:tasks.rados.rados.0.smithi017.stdout:1196: oid 6 version 127 is already newer than 127 2023-03-29T14:07:23.827 INFO:tasks.rados.rados.0.smithi017.stdout:1196: oid 6 version 127 is already newer than 127 2023-03-29T14:07:23.827 INFO:tasks.rados.rados.0.smithi017.stdout:1196: oid 6 version 127 is already newer than 123 2023-03-29T14:07:23.827 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 127 (ObjNum 581 snap 137 seq_num 581) dirty exists 2023-03-29T14:07:23.827 INFO:tasks.rados.rados.0.smithi017.stdout:1196: left oid 6 (ObjNum 581 snap 137 seq_num 581) 2023-03-29T14:07:24.251 INFO:tasks.rados.rados.0.smithi017.stdout:1196: done (1 left) 2023-03-29T14:07:24.251 INFO:tasks.rados.rados.0.smithi017.stdout:1197: done (0 left) 2023-03-29T14:07:24.252 INFO:tasks.rados.rados.0.smithi017.stdout:1198: read oid 31 snap 132 2023-03-29T14:07:24.252 INFO:tasks.rados.rados.0.smithi017.stdout:1198: expect deleted 2023-03-29T14:07:24.252 INFO:tasks.rados.rados.0.smithi017.stdout:1199: write (excl) oid 45 current snap is 138 2023-03-29T14:07:24.252 INFO:tasks.rados.rados.0.smithi017.stdout:1199: seq_num 582 ranges {581674=637746,1840873=558854,3066417=537135} 2023-03-29T14:07:24.260 INFO:tasks.rados.rados.0.smithi017.stdout:1199: writing smithi017107884-45 from 581674 to 1219420 tid 1 2023-03-29T14:07:24.269 INFO:tasks.rados.rados.0.smithi017.stdout:1199: writing smithi017107884-45 from 1840873 to 2399727 tid 2 2023-03-29T14:07:24.278 INFO:tasks.rados.rados.0.smithi017.stdout:1199: writing smithi017107884-45 from 3066417 to 3603552 tid 3 2023-03-29T14:07:24.278 INFO:tasks.rados.rados.0.smithi017.stdout:1200: write (excl) oid 17 current snap is 138 2023-03-29T14:07:24.278 INFO:tasks.rados.rados.0.smithi017.stdout:1200: seq_num 583 ranges {598481=712995,1899355=418865,3093840=704641} 2023-03-29T14:07:24.288 INFO:tasks.rados.rados.0.smithi017.stdout:1200: writing smithi017107884-17 from 598481 to 1311476 tid 1 2023-03-29T14:07:24.295 INFO:tasks.rados.rados.0.smithi017.stdout:1200: writing smithi017107884-17 from 1899355 to 2318220 tid 2 2023-03-29T14:07:24.306 INFO:tasks.rados.rados.0.smithi017.stdout:1200: writing smithi017107884-17 from 3093840 to 3798481 tid 3 2023-03-29T14:07:24.307 INFO:tasks.rados.rados.0.smithi017.stdout:1201: write oid 2 current snap is 138 2023-03-29T14:07:24.307 INFO:tasks.rados.rados.0.smithi017.stdout:1201: seq_num 584 ranges {615288=788244,1957837=562523,3121263=559679} 2023-03-29T14:07:24.317 INFO:tasks.rados.rados.0.smithi017.stdout:1201: writing smithi017107884-2 from 615288 to 1403532 tid 1 2023-03-29T14:07:24.326 INFO:tasks.rados.rados.0.smithi017.stdout:1201: writing smithi017107884-2 from 1957837 to 2520360 tid 2 2023-03-29T14:07:24.334 INFO:tasks.rados.rados.0.smithi017.stdout:1201: writing smithi017107884-2 from 3121263 to 3680942 tid 3 2023-03-29T14:07:24.335 INFO:tasks.rados.rados.0.smithi017.stdout:1202: write (excl) oid 33 current snap is 138 2023-03-29T14:07:24.335 INFO:tasks.rados.rados.0.smithi017.stdout:1202: seq_num 585 ranges {632095=463493,1616319=422534,2748686=487304} 2023-03-29T14:07:24.342 INFO:tasks.rados.rados.0.smithi017.stdout:1202: writing smithi017107884-33 from 632095 to 1095588 tid 1 2023-03-29T14:07:24.349 INFO:tasks.rados.rados.0.smithi017.stdout:1202: writing smithi017107884-33 from 1616319 to 2038853 tid 2 2023-03-29T14:07:24.358 INFO:tasks.rados.rados.0.smithi017.stdout:1202: writing smithi017107884-33 from 2748686 to 3235990 tid 3 2023-03-29T14:07:24.358 INFO:tasks.rados.rados.0.smithi017.stdout:1203: snap_remove snap 136 2023-03-29T14:07:24.598 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:24.725+0000 7f9ce0c8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:24.729+0000 7f9ce0c8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:24.795 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:24.795 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1203: done (5 left) 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1204: read oid 42 snap -1 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1204: expect deleted 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1205: read oid 22 snap -1 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1205: expect (ObjNum 553 snap 127 seq_num 553) 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1206: delete oid 44 current snap is 138 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1201: finishing write tid 1 to smithi017107884-2 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1201: finishing write tid 2 to smithi017107884-2 2023-03-29T14:07:25.255 INFO:tasks.rados.rados.0.smithi017.stdout:1201: finishing write tid 3 to smithi017107884-2 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: finishing write tid 1 to smithi017107884-17 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1201: finishing write tid 4 to smithi017107884-2 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1201: finishing write tid 5 to smithi017107884-2 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1201: oid 2 updating version 0 to 253 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1201: oid 2 version 253 is already newer than 253 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1201: oid 2 version 253 is already newer than 253 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1201: oid 2 version 253 is already newer than 252 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 253 (ObjNum 584 snap 138 seq_num 584) dirty exists 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1201: left oid 2 (ObjNum 584 snap 138 seq_num 584) 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1199: finishing write tid 1 to smithi017107884-45 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1202: finishing write tid 1 to smithi017107884-33 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: finishing write tid 2 to smithi017107884-17 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: finishing write tid 3 to smithi017107884-17 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: finishing write tid 4 to smithi017107884-17 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: finishing write tid 5 to smithi017107884-17 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: oid 17 updating version 0 to 184 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: oid 17 updating version 184 to 187 2023-03-29T14:07:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1200: oid 17 version 187 is already newer than 187 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1200: oid 17 version 187 is already newer than 187 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 187 (ObjNum 583 snap 138 seq_num 583) dirty exists 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1200: left oid 17 (ObjNum 583 snap 138 seq_num 583) 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1199: finishing write tid 2 to smithi017107884-45 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: finishing write tid 2 to smithi017107884-33 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: finishing write tid 3 to smithi017107884-33 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: finishing write tid 4 to smithi017107884-33 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: finishing write tid 5 to smithi017107884-33 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: oid 33 updating version 0 to 162 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: oid 33 updating version 162 to 165 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: oid 33 version 165 is already newer than 165 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: oid 33 version 165 is already newer than 165 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 165 (ObjNum 585 snap 138 seq_num 585) dirty exists 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1202: left oid 33 (ObjNum 585 snap 138 seq_num 585) 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1199: finishing write tid 3 to smithi017107884-45 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1199: finishing write tid 4 to smithi017107884-45 2023-03-29T14:07:25.257 INFO:tasks.rados.rados.0.smithi017.stdout:1199: finishing write tid 5 to smithi017107884-45 2023-03-29T14:07:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:1199: oid 45 updating version 0 to 177 2023-03-29T14:07:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:1199: oid 45 updating version 177 to 181 2023-03-29T14:07:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:1199: oid 45 version 181 is already newer than 181 2023-03-29T14:07:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:1199: oid 45 version 181 is already newer than 181 2023-03-29T14:07:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 181 (ObjNum 582 snap 138 seq_num 582) dirty exists 2023-03-29T14:07:25.258 INFO:tasks.rados.rados.0.smithi017.stdout:1199: left oid 45 (ObjNum 582 snap 138 seq_num 582) 2023-03-29T14:07:25.296 INFO:tasks.rados.rados.0.smithi017.stdout:1205: expect (ObjNum 553 snap 127 seq_num 553) 2023-03-29T14:07:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:1198: done (7 left) 2023-03-29T14:07:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:1199: done (6 left) 2023-03-29T14:07:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:1200: done (5 left) 2023-03-29T14:07:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:1201: done (4 left) 2023-03-29T14:07:25.535 INFO:tasks.rados.rados.0.smithi017.stdout:1202: done (3 left) 2023-03-29T14:07:25.536 INFO:tasks.rados.rados.0.smithi017.stdout:1204: done (2 left) 2023-03-29T14:07:25.536 INFO:tasks.rados.rados.0.smithi017.stdout:1205: done (1 left) 2023-03-29T14:07:25.536 INFO:tasks.rados.rados.0.smithi017.stdout:1206: done (0 left) 2023-03-29T14:07:25.536 INFO:tasks.rados.rados.0.smithi017.stdout:1207: read oid 28 snap -1 2023-03-29T14:07:25.536 INFO:tasks.rados.rados.0.smithi017.stdout:1207: expect deleted 2023-03-29T14:07:25.536 INFO:tasks.rados.rados.0.smithi017.stdout:1208: snap_create 2023-03-29T14:07:25.661 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:25.661 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:25.661 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:25.792 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:25.787+0000 7ff074cd6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:25.795 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:25.794+0000 7ff074cd6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:26.009 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:26.011 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:26.022 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2023-03-29T14:07:25.253589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197458954,"num_objects":652,"num_object_clones":556,"num_object_copies":1956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":2710,"num_read_kb":1534556,"num_write":3742,"num_write_kb":1012268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1964,"ondisk_log_size":1964,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1297004,"kb_used_data":1231756,"kb_used_omap":0,"kb_used_meta":65216,"kb_avail":376190356,"statfs":{"total":386547056640,"available":385218924544,"internally_reserved":0,"allocated":1261318144,"data_stored":1530535502,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":66781184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7664190,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":16,"num_read_kb":15305,"num_write":20,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017151"},"pg_stats":[{"pgid":"3.f","version":"277'88","reported_seq":218,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:20.254657+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:20.254657+0000","last_peered":"2023-03-29T14:07:20.254657+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:20.254657+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12235892,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":34315,"num_write":98,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":307,"reported_epoch":291,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:17.260662+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:17.260662+0000","last_peered":"2023-03-29T14:07:17.260662+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:17.260662+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"293'183","reported_seq":644,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-03-29T14:07:20.262181+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:20.262181+0000","last_peered":"2023-03-29T14:07:20.262181+0000","last_clean":"2023-03-29T14:07:20.262181+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:20.262181+0000","last_undegraded":"2023-03-29T14:07:20.262181+0000","last_fullsized":"2023-03-29T14:07:20.262181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27597886,"num_objects":62,"num_object_clones":52,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":247,"num_read_kb":157795,"num_write":348,"num_write_kb":103470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"288'167","reported_seq":624,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-03-29T14:07:20.248319+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:20.248319+0000","last_peered":"2023-03-29T14:07:20.248319+0000","last_clean":"2023-03-29T14:07:20.248319+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:20.248319+0000","last_undegraded":"2023-03-29T14:07:20.248319+0000","last_fullsized":"2023-03-29T14:07:20.248319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18521908,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":261,"num_read_kb":158287,"num_write":328,"num_write_kb":92538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"284'176","reported_seq":610,"reported_epoch":291,"state":"active+clean","last_fresh":"2023-03-29T14:07:17.257236+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:17.257236+0000","last_peered":"2023-03-29T14:07:17.257236+0000","last_clean":"2023-03-29T14:07:17.257236+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:17.257236+0000","last_undegraded":"2023-03-29T14:07:17.257236+0000","last_fullsized":"2023-03-29T14:07:17.257236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12303120,"num_objects":66,"num_object_clones":58,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":218,"num_read_kb":120996,"num_write":354,"num_write_kb":102714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":245,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-03-29T14:07:20.259609+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:20.259609+0000","last_peered":"2023-03-29T14:07:20.259609+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:20.259609+0000","last_undegraded":"2023-03-29T14:07:20.259609+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"260'104","reported_seq":467,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:20.249507+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:20.249507+0000","last_peered":"2023-03-29T14:07:20.249507+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:20.249507+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748192,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":127,"num_read_kb":61239,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"255'69","reported_seq":398,"reported_epoch":295,"state":"active+clean","last_fresh":"2023-03-29T14:07:20.257119+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:20.257119+0000","last_peered":"2023-03-29T14:07:20.257119+0000","last_clean":"2023-03-29T14:07:20.257119+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:20.257119+0000","last_undegraded":"2023-03-29T14:07:20.257119+0000","last_fullsized":"2023-03-29T14:07:20.257119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5512894,"num_objects":28,"num_object_clones":24,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":102,"num_read_kb":57965,"num_write":134,"num_write_kb":38116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":230,"reported_epoch":291,"state":"active+undersized","last_fresh":"2023-03-29T14:07:17.239298+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:17.239298+0000","last_peered":"2023-03-29T14:07:17.239298+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:17.239298+0000","last_undegraded":"2023-03-29T14:07:17.239298+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"273'65","reported_seq":151,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:17.489180+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:17.489180+0000","last_peered":"2023-03-29T14:07:17.489180+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:17.489180+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":30,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":248,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-03-29T14:07:20.253963+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:20.253963+0000","last_peered":"2023-03-29T14:07:20.253963+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:20.253963+0000","last_undegraded":"2023-03-29T14:07:20.253963+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":417,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:17.493187+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:17.493187+0000","last_peered":"2023-03-29T14:07:17.493187+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:17.493187+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":91,"reported_epoch":291,"state":"active+undersized","last_fresh":"2023-03-29T14:07:17.233588+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:17.233588+0000","last_peered":"2023-03-29T14:07:17.233588+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:17.233588+0000","last_undegraded":"2023-03-29T14:07:17.233588+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"286'105","reported_seq":455,"reported_epoch":291,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:17.272744+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:17.272744+0000","last_peered":"2023-03-29T14:07:17.272744+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:17.272744+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13815750,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":58665,"num_write":208,"num_write_kb":61634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":248,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-03-29T14:07:20.252665+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:20.252665+0000","last_peered":"2023-03-29T14:07:20.252665+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:20.252665+0000","last_undegraded":"2023-03-29T14:07:20.252665+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":455,"reported_epoch":291,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:17.244928+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:17.244928+0000","last_peered":"2023-03-29T14:07:17.244928+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:17.244928+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"286'122","reported_seq":566,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:20.250855+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:20.250855+0000","last_peered":"2023-03-29T14:07:20.250855+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:20.250855+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17547102,"num_objects":42,"num_object_clones":34,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":259,"num_read_kb":179139,"num_write":242,"num_write_kb":69906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":42}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":248,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-03-29T14:07:20.258739+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:20.258739+0000","last_peered":"2023-03-29T14:07:20.258739+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:20.258739+0000","last_undegraded":"2023-03-29T14:07:20.258739+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"291'242","reported_seq":797,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:19.422297+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:19.422297+0000","last_peered":"2023-03-29T14:07:19.422297+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:19.422297+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37144658,"num_objects":82,"num_object_clones":68,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":397,"num_read_kb":247571,"num_write":456,"num_write_kb":131810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":82}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":230,"reported_epoch":291,"state":"active+undersized","last_fresh":"2023-03-29T14:07:17.269357+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:17.269357+0000","last_peered":"2023-03-29T14:07:17.269357+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:17.269357+0000","last_undegraded":"2023-03-29T14:07:17.269357+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"286'85","reported_seq":418,"reported_epoch":291,"state":"active+clean","last_fresh":"2023-03-29T14:07:17.266249+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:17.266249+0000","last_peered":"2023-03-29T14:07:17.266249+0000","last_clean":"2023-03-29T14:07:17.266249+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:17.266249+0000","last_undegraded":"2023-03-29T14:07:17.266249+0000","last_fullsized":"2023-03-29T14:07:17.266249+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7092752,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":122,"num_read_kb":70178,"num_write":176,"num_write_kb":50470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"282'184","reported_seq":621,"reported_epoch":291,"state":"active+clean","last_fresh":"2023-03-29T14:07:17.223523+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:17.223523+0000","last_peered":"2023-03-29T14:07:17.223523+0000","last_clean":"2023-03-29T14:07:17.223523+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:17.223523+0000","last_undegraded":"2023-03-29T14:07:17.223523+0000","last_fullsized":"2023-03-29T14:07:17.223523+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":229,"num_read_kb":133906,"num_write":344,"num_write_kb":100514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":230,"reported_epoch":291,"state":"active+undersized","last_fresh":"2023-03-29T14:07:17.262547+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:17.262547+0000","last_peered":"2023-03-29T14:07:17.262547+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:17.262547+0000","last_undegraded":"2023-03-29T14:07:17.262547+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":248,"reported_epoch":295,"state":"active+undersized","last_fresh":"2023-03-29T14:07:20.260844+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:20.260844+0000","last_peered":"2023-03-29T14:07:20.260844+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:20.260844+0000","last_undegraded":"2023-03-29T14:07:20.260844+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"287'161","reported_seq":587,"reported_epoch":295,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:17.499036+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:17.499036+0000","last_peered":"2023-03-29T14:07:17.499036+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:17.499036+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12437576,"num_objects":58,"num_object_clones":50,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":211,"num_read_kb":127825,"num_write":310,"num_write_kb":93760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":58}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":196278218,"num_objects":648,"num_object_clones":556,"num_object_copies":1944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":328,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":36,"num_read":2568,"num_read_kb":1534436,"num_write":3468,"num_write_kb":1008480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1867,"ondisk_log_size":1867,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":360044,"kb_used_data":344292,"kb_used_omap":0,"kb_used_meta":15744,"kb_avail":94011796,"statfs":{"total":96636764160,"available":96268079104,"internally_reserved":0,"allocated":352555008,"data_stored":426508569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16121856},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":412804,"kb_used_data":391740,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":93959036,"statfs":{"total":96636764160,"available":96214052864,"internally_reserved":0,"allocated":401141760,"data_stored":491037270,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":368212,"kb_used_data":351884,"kb_used_omap":0,"kb_used_meta":16320,"kb_avail":94003628,"statfs":{"total":96636764160,"available":96259715072,"internally_reserved":0,"allocated":360329216,"data_stored":439757438,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":16711680},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:26.028 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.9 3.5 3.0 1.0 3.2 3.1 2023-03-29T14:07:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:26.155+0000 7fa3cf3ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:26.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:26.171+0000 7fa3cf3ef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:26.259 INFO:tasks.rados.rados.0.smithi017.stdout:1207: done (1 left) 2023-03-29T14:07:26.259 INFO:tasks.rados.rados.0.smithi017.stdout:1208: done (0 left) 2023-03-29T14:07:26.259 INFO:tasks.rados.rados.0.smithi017.stdout:1209: write (excl) oid 7 current snap is 139 2023-03-29T14:07:26.259 INFO:tasks.rados.rados.0.smithi017.stdout:1209: seq_num 587 ranges {665709=730344,2133283=426203,3203532=509848} 2023-03-29T14:07:26.269 INFO:tasks.rados.rados.0.smithi017.stdout:1209: writing smithi017107884-7 from 665709 to 1396053 tid 1 2023-03-29T14:07:26.277 INFO:tasks.rados.rados.0.smithi017.stdout:1209: writing smithi017107884-7 from 2133283 to 2559486 tid 2 2023-03-29T14:07:26.285 INFO:tasks.rados.rados.0.smithi017.stdout:1209: writing smithi017107884-7 from 3203532 to 3713380 tid 3 2023-03-29T14:07:26.285 INFO:tasks.rados.rados.0.smithi017.stdout:1210: delete oid 36 current snap is 139 2023-03-29T14:07:26.375 INFO:tasks.rados.rados.0.smithi017.stdout:1209: finishing write tid 1 to smithi017107884-7 2023-03-29T14:07:26.382 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.5] on osd.0 to force-recovery; instructing pg(s) [1.0,3.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.2,3.9,3.f] on osd.3 to force-recovery; 2023-03-29T14:07:26.382 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:07:26.408 INFO:tasks.rados.rados.0.smithi017.stdout:1210: done (1 left) 2023-03-29T14:07:26.408 INFO:tasks.rados.rados.0.smithi017.stdout:1211: write (excl) oid 8 current snap is 139 2023-03-29T14:07:26.408 INFO:tasks.rados.rados.0.smithi017.stdout:1211: seq_num 589 ranges {699323=480842,1850247=429872,2858378=532392} 2023-03-29T14:07:26.416 INFO:tasks.rados.rados.0.smithi017.stdout:1211: writing smithi017107884-8 from 699323 to 1180165 tid 1 2023-03-29T14:07:26.424 INFO:tasks.rados.rados.0.smithi017.stdout:1211: writing smithi017107884-8 from 1850247 to 2280119 tid 2 2023-03-29T14:07:26.432 INFO:tasks.rados.rados.0.smithi017.stdout:1211: writing smithi017107884-8 from 2858378 to 3390770 tid 3 2023-03-29T14:07:26.432 INFO:tasks.rados.rados.0.smithi017.stdout:1212: delete oid 14 current snap is 139 2023-03-29T14:07:26.434 INFO:tasks.rados.rados.0.smithi017.stdout:1212: done (2 left) 2023-03-29T14:07:26.434 INFO:tasks.rados.rados.0.smithi017.stdout:1213: write oid 19 current snap is 139 2023-03-29T14:07:26.434 INFO:tasks.rados.rados.0.smithi017.stdout:1213: seq_num 591 ranges {732937=631340,1850858=433541,2796871=671289,3888195=44742} 2023-03-29T14:07:26.444 INFO:tasks.rados.rados.0.smithi017.stdout:1213: writing smithi017107884-19 from 732937 to 1364277 tid 1 2023-03-29T14:07:26.451 INFO:tasks.rados.rados.0.smithi017.stdout:1213: writing smithi017107884-19 from 1850858 to 2284399 tid 2 2023-03-29T14:07:26.460 INFO:tasks.rados.rados.0.smithi017.stdout:1213: writing smithi017107884-19 from 2796871 to 3468160 tid 3 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1213: writing smithi017107884-19 from 3888195 to 3932937 tid 4 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1214: read oid 31 snap -1 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1214: expect deleted 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1215: read oid 6 snap -1 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1215: expect (ObjNum 581 snap 137 seq_num 581) 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1216: write oid 4 current snap is 139 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1209: finishing write tid 2 to smithi017107884-7 2023-03-29T14:07:26.463 INFO:tasks.rados.rados.0.smithi017.stdout:1216: seq_num 592 ranges {749744=706589,1909340=577199,3224294=482561} 2023-03-29T14:07:26.473 INFO:tasks.rados.rados.0.smithi017.stdout:1216: writing smithi017107884-4 from 749744 to 1456333 tid 1 2023-03-29T14:07:26.481 INFO:tasks.rados.rados.0.smithi017.stdout:1216: writing smithi017107884-4 from 1909340 to 2486539 tid 2 2023-03-29T14:07:26.491 INFO:tasks.rados.rados.0.smithi017.stdout:1216: writing smithi017107884-4 from 3224294 to 3706855 tid 3 2023-03-29T14:07:26.491 INFO:tasks.rados.rados.0.smithi017.stdout:1217: read oid 37 snap 137 2023-03-29T14:07:26.491 INFO:tasks.rados.rados.0.smithi017.stdout:1217: expect deleted 2023-03-29T14:07:26.491 INFO:tasks.rados.rados.0.smithi017.stdout:1218: snap_create 2023-03-29T14:07:26.491 INFO:tasks.rados.rados.0.smithi017.stdout:1209: finishing write tid 3 to smithi017107884-7 2023-03-29T14:07:26.491 INFO:tasks.rados.rados.0.smithi017.stdout:1209: finishing write tid 4 to smithi017107884-7 2023-03-29T14:07:26.491 INFO:tasks.rados.rados.0.smithi017.stdout:1209: finishing write tid 5 to smithi017107884-7 2023-03-29T14:07:26.492 INFO:tasks.rados.rados.0.smithi017.stdout:1209: oid 7 updating version 0 to 72 2023-03-29T14:07:26.492 INFO:tasks.rados.rados.0.smithi017.stdout:1209: oid 7 updating version 72 to 75 2023-03-29T14:07:26.492 INFO:tasks.rados.rados.0.smithi017.stdout:1209: oid 7 version 75 is already newer than 75 2023-03-29T14:07:26.492 INFO:tasks.rados.rados.0.smithi017.stdout:1209: oid 7 version 75 is already newer than 75 2023-03-29T14:07:26.492 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 75 (ObjNum 587 snap 139 seq_num 587) dirty exists 2023-03-29T14:07:26.492 INFO:tasks.rados.rados.0.smithi017.stdout:1209: left oid 7 (ObjNum 587 snap 139 seq_num 587) 2023-03-29T14:07:26.492 INFO:tasks.rados.rados.0.smithi017.stdout:1215: expect (ObjNum 581 snap 137 seq_num 581) 2023-03-29T14:07:26.698 INFO:tasks.rados.rados.0.smithi017.stdout:1211: finishing write tid 1 to smithi017107884-8 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1216: finishing write tid 1 to smithi017107884-4 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: finishing write tid 2 to smithi017107884-8 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1216: finishing write tid 2 to smithi017107884-4 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: finishing write tid 3 to smithi017107884-8 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: finishing write tid 4 to smithi017107884-8 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: finishing write tid 5 to smithi017107884-8 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: oid 8 updating version 0 to 170 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: oid 8 version 170 is already newer than 170 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: oid 8 version 170 is already newer than 166 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: oid 8 version 170 is already newer than 170 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 170 (ObjNum 589 snap 139 seq_num 589) dirty exists 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1211: left oid 8 (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1216: finishing write tid 3 to smithi017107884-4 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1216: finishing write tid 4 to smithi017107884-4 2023-03-29T14:07:26.699 INFO:tasks.rados.rados.0.smithi017.stdout:1216: finishing write tid 5 to smithi017107884-4 2023-03-29T14:07:26.700 INFO:tasks.rados.rados.0.smithi017.stdout:1216: oid 4 updating version 0 to 192 2023-03-29T14:07:26.700 INFO:tasks.rados.rados.0.smithi017.stdout:1216: oid 4 version 192 is already newer than 192 2023-03-29T14:07:26.700 INFO:tasks.rados.rados.0.smithi017.stdout:1216: oid 4 version 192 is already newer than 192 2023-03-29T14:07:26.700 INFO:tasks.rados.rados.0.smithi017.stdout:1216: oid 4 version 192 is already newer than 192 2023-03-29T14:07:26.700 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 192 (ObjNum 592 snap 139 seq_num 592) dirty exists 2023-03-29T14:07:26.700 INFO:tasks.rados.rados.0.smithi017.stdout:1216: left oid 4 (ObjNum 592 snap 139 seq_num 592) 2023-03-29T14:07:26.716 INFO:tasks.rados.rados.0.smithi017.stdout:1213: finishing write tid 1 to smithi017107884-19 2023-03-29T14:07:26.722 INFO:tasks.rados.rados.0.smithi017.stdout:1213: finishing write tid 2 to smithi017107884-19 2023-03-29T14:07:26.725 INFO:tasks.rados.rados.0.smithi017.stdout:1213: finishing write tid 3 to smithi017107884-19 2023-03-29T14:07:26.726 INFO:tasks.rados.rados.0.smithi017.stdout:1213: finishing write tid 4 to smithi017107884-19 2023-03-29T14:07:26.728 INFO:tasks.rados.rados.0.smithi017.stdout:1213: finishing write tid 5 to smithi017107884-19 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:1213: finishing write tid 6 to smithi017107884-19 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:1213: oid 19 updating version 0 to 175 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:1213: oid 19 version 175 is already newer than 175 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:1213: oid 19 version 175 is already newer than 175 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:1213: oid 19 version 175 is already newer than 175 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:1213: oid 19 version 175 is already newer than 175 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 175 (ObjNum 591 snap 139 seq_num 591) dirty exists 2023-03-29T14:07:26.731 INFO:tasks.rados.rados.0.smithi017.stdout:1213: left oid 19 (ObjNum 591 snap 139 seq_num 591) 2023-03-29T14:07:26.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:26.923+0000 7fd31b99b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:26.927+0000 7fd31b99b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:26.993 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:26.994 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:1209: done (7 left) 2023-03-29T14:07:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:1211: done (6 left) 2023-03-29T14:07:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:1213: done (5 left) 2023-03-29T14:07:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:1214: done (4 left) 2023-03-29T14:07:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:1215: done (3 left) 2023-03-29T14:07:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:1216: done (2 left) 2023-03-29T14:07:27.262 INFO:tasks.rados.rados.0.smithi017.stdout:1217: done (1 left) 2023-03-29T14:07:27.263 INFO:tasks.rados.rados.0.smithi017.stdout:1218: done (0 left) 2023-03-29T14:07:27.263 INFO:tasks.rados.rados.0.smithi017.stdout:1219: delete oid 41 current snap is 140 2023-03-29T14:07:27.334 INFO:tasks.rados.rados.0.smithi017.stdout:1219: done (0 left) 2023-03-29T14:07:27.334 INFO:tasks.rados.rados.0.smithi017.stdout:1220: read oid 43 snap -1 2023-03-29T14:07:27.334 INFO:tasks.rados.rados.0.smithi017.stdout:1220: expect (ObjNum 571 snap 132 seq_num 571) 2023-03-29T14:07:27.334 INFO:tasks.rados.rados.0.smithi017.stdout:1221: delete oid 18 current snap is 140 2023-03-29T14:07:27.362 INFO:tasks.rados.rados.0.smithi017.stdout:1220: expect (ObjNum 571 snap 132 seq_num 571) 2023-03-29T14:07:27.551 INFO:tasks.rados.rados.0.smithi017.stdout:1220: done (1 left) 2023-03-29T14:07:27.552 INFO:tasks.rados.rados.0.smithi017.stdout:1221: done (0 left) 2023-03-29T14:07:27.552 INFO:tasks.rados.rados.0.smithi017.stdout:1222: read oid 18 snap 137 2023-03-29T14:07:27.552 INFO:tasks.rados.rados.0.smithi017.stdout:1222: expect (ObjNum 528 snap 120 seq_num 528) 2023-03-29T14:07:27.552 INFO:tasks.rados.rados.0.smithi017.stdout:1223: write oid 35 current snap is 140 2023-03-29T14:07:27.552 INFO:tasks.rados.rados.0.smithi017.stdout:1223: seq_num 595 ranges {400165=648689,1684786=315379} 2023-03-29T14:07:27.559 INFO:tasks.rados.rados.0.smithi017.stdout:1223: writing smithi017107884-35 from 400165 to 1048854 tid 1 2023-03-29T14:07:27.566 INFO:tasks.rados.rados.0.smithi017.stdout:1223: writing smithi017107884-35 from 1684786 to 2000165 tid 2 2023-03-29T14:07:27.566 INFO:tasks.rados.rados.0.smithi017.stdout:1224: snap_create 2023-03-29T14:07:27.566 INFO:tasks.rados.rados.0.smithi017.stdout:1222: expect (ObjNum 528 snap 120 seq_num 528) 2023-03-29T14:07:27.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~138s 2023-03-29T14:07:27.682 INFO:tasks.rados.rados.0.smithi017.stdout:1223: finishing write tid 1 to smithi017107884-35 2023-03-29T14:07:27.682 INFO:tasks.rados.rados.0.smithi017.stdout:1223: finishing write tid 2 to smithi017107884-35 2023-03-29T14:07:27.682 INFO:tasks.rados.rados.0.smithi017.stdout:1223: finishing write tid 3 to smithi017107884-35 2023-03-29T14:07:27.683 INFO:tasks.rados.rados.0.smithi017.stdout:1223: finishing write tid 4 to smithi017107884-35 2023-03-29T14:07:27.683 INFO:tasks.rados.rados.0.smithi017.stdout:1223: oid 35 updating version 0 to 90 2023-03-29T14:07:27.683 INFO:tasks.rados.rados.0.smithi017.stdout:1223: oid 35 version 90 is already newer than 90 2023-03-29T14:07:27.683 INFO:tasks.rados.rados.0.smithi017.stdout:1223: oid 35 version 90 is already newer than 90 2023-03-29T14:07:27.683 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 90 (ObjNum 595 snap 140 seq_num 595) dirty exists 2023-03-29T14:07:27.683 INFO:tasks.rados.rados.0.smithi017.stdout:1223: left oid 35 (ObjNum 595 snap 140 seq_num 595) 2023-03-29T14:07:28.235 INFO:tasks.rados.rados.0.smithi017.stdout:1222: done (2 left) 2023-03-29T14:07:28.235 INFO:tasks.rados.rados.0.smithi017.stdout:1223: done (1 left) 2023-03-29T14:07:28.235 INFO:tasks.rados.rados.0.smithi017.stdout:1224: done (0 left) 2023-03-29T14:07:28.235 INFO:tasks.rados.rados.0.smithi017.stdout:1225: read oid 28 snap 138 2023-03-29T14:07:28.235 INFO:tasks.rados.rados.0.smithi017.stdout:1225: expect deleted 2023-03-29T14:07:28.235 INFO:tasks.rados.rados.0.smithi017.stdout:1226: snap_remove snap 140 2023-03-29T14:07:28.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:29.123 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:29.121+0000 7f6360b95700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:29.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:29.126+0000 7f6360b95700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:29.192 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:29.193 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1226: done (1 left) 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1227: read oid 46 snap -1 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1227: expect deleted 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1228: read oid 14 snap -1 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1228: expect deleted 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1229: read oid 29 snap 139 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1229: expect (ObjNum 429 snap 98 seq_num 429) 2023-03-29T14:07:29.239 INFO:tasks.rados.rados.0.smithi017.stdout:1230: delete oid 1 current snap is 141 2023-03-29T14:07:29.348 INFO:tasks.rados.rados.0.smithi017.stdout:1229: expect (ObjNum 429 snap 98 seq_num 429) 2023-03-29T14:07:29.510 INFO:tasks.rados.rados.0.smithi017.stdout:1225: done (4 left) 2023-03-29T14:07:29.511 INFO:tasks.rados.rados.0.smithi017.stdout:1227: done (3 left) 2023-03-29T14:07:29.511 INFO:tasks.rados.rados.0.smithi017.stdout:1228: done (2 left) 2023-03-29T14:07:29.511 INFO:tasks.rados.rados.0.smithi017.stdout:1229: done (1 left) 2023-03-29T14:07:29.511 INFO:tasks.rados.rados.0.smithi017.stdout:1230: done (0 left) 2023-03-29T14:07:29.511 INFO:tasks.rados.rados.0.smithi017.stdout:1231: write oid 22 current snap is 141 2023-03-29T14:07:29.511 INFO:tasks.rados.rados.0.smithi017.stdout:1231: seq_num 597 ranges {433779=799187,1801750=232029} 2023-03-29T14:07:29.520 INFO:tasks.rados.rados.0.smithi017.stdout:1231: writing smithi017107884-22 from 433779 to 1232966 tid 1 2023-03-29T14:07:29.525 INFO:tasks.rados.rados.0.smithi017.stdout:1231: writing smithi017107884-22 from 1801750 to 2033779 tid 2 2023-03-29T14:07:29.525 INFO:tasks.rados.rados.0.smithi017.stdout:1232: write oid 21 current snap is 141 2023-03-29T14:07:29.525 INFO:tasks.rados.rados.0.smithi017.stdout:1232: seq_num 598 ranges {450586=474436,1343879=588206} 2023-03-29T14:07:29.532 INFO:tasks.rados.rados.0.smithi017.stdout:1232: writing smithi017107884-21 from 450586 to 925022 tid 1 2023-03-29T14:07:29.540 INFO:tasks.rados.rados.0.smithi017.stdout:1232: writing smithi017107884-21 from 1343879 to 1932085 tid 2 2023-03-29T14:07:29.540 INFO:tasks.rados.rados.0.smithi017.stdout:1233: snap_create 2023-03-29T14:07:29.567 INFO:tasks.rados.rados.0.smithi017.stdout:1231: finishing write tid 1 to smithi017107884-22 2023-03-29T14:07:29.576 INFO:tasks.rados.rados.0.smithi017.stdout:1232: finishing write tid 1 to smithi017107884-21 2023-03-29T14:07:29.613 INFO:tasks.rados.rados.0.smithi017.stdout:1231: finishing write tid 2 to smithi017107884-22 2023-03-29T14:07:29.615 INFO:tasks.rados.rados.0.smithi017.stdout:1231: finishing write tid 3 to smithi017107884-22 2023-03-29T14:07:29.615 INFO:tasks.rados.rados.0.smithi017.stdout:1231: finishing write tid 4 to smithi017107884-22 2023-03-29T14:07:29.616 INFO:tasks.rados.rados.0.smithi017.stdout:1231: oid 22 updating version 0 to 92 2023-03-29T14:07:29.616 INFO:tasks.rados.rados.0.smithi017.stdout:1231: oid 22 version 92 is already newer than 92 2023-03-29T14:07:29.616 INFO:tasks.rados.rados.0.smithi017.stdout:1231: oid 22 version 92 is already newer than 92 2023-03-29T14:07:29.616 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 92 (ObjNum 597 snap 141 seq_num 597) dirty exists 2023-03-29T14:07:29.616 INFO:tasks.rados.rados.0.smithi017.stdout:1231: left oid 22 (ObjNum 597 snap 141 seq_num 597) 2023-03-29T14:07:29.622 INFO:tasks.rados.rados.0.smithi017.stdout:1232: finishing write tid 2 to smithi017107884-21 2023-03-29T14:07:29.624 INFO:tasks.rados.rados.0.smithi017.stdout:1232: finishing write tid 3 to smithi017107884-21 2023-03-29T14:07:29.625 INFO:tasks.rados.rados.0.smithi017.stdout:1232: finishing write tid 4 to smithi017107884-21 2023-03-29T14:07:29.625 INFO:tasks.rados.rados.0.smithi017.stdout:1232: oid 21 updating version 0 to 131 2023-03-29T14:07:29.625 INFO:tasks.rados.rados.0.smithi017.stdout:1232: oid 21 version 131 is already newer than 131 2023-03-29T14:07:29.625 INFO:tasks.rados.rados.0.smithi017.stdout:1232: oid 21 version 131 is already newer than 131 2023-03-29T14:07:29.625 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 131 (ObjNum 598 snap 141 seq_num 598) dirty exists 2023-03-29T14:07:29.625 INFO:tasks.rados.rados.0.smithi017.stdout:1232: left oid 21 (ObjNum 598 snap 141 seq_num 598) 2023-03-29T14:07:30.269 INFO:tasks.rados.rados.0.smithi017.stdout:1231: done (2 left) 2023-03-29T14:07:30.270 INFO:tasks.rados.rados.0.smithi017.stdout:1232: done (1 left) 2023-03-29T14:07:30.270 INFO:tasks.rados.rados.0.smithi017.stdout:1233: done (0 left) 2023-03-29T14:07:30.270 INFO:tasks.rados.rados.0.smithi017.stdout:1234: write oid 32 current snap is 142 2023-03-29T14:07:30.270 INFO:tasks.rados.rados.0.smithi017.stdout:1234: seq_num 599 ranges {467393=549685,1802361=265032} 2023-03-29T14:07:30.277 INFO:tasks.rados.rados.0.smithi017.stdout:1234: writing smithi017107884-32 from 467393 to 1017078 tid 1 2023-03-29T14:07:30.284 INFO:tasks.rados.rados.0.smithi017.stdout:1234: writing smithi017107884-32 from 1802361 to 2067393 tid 2 2023-03-29T14:07:30.284 INFO:tasks.rados.rados.0.smithi017.stdout:1235: delete oid 34 current snap is 142 2023-03-29T14:07:30.361 INFO:tasks.rados.rados.0.smithi017.stdout:1235: done (1 left) 2023-03-29T14:07:30.361 INFO:tasks.rados.rados.0.smithi017.stdout:1236: read oid 20 snap -1 2023-03-29T14:07:30.361 INFO:tasks.rados.rados.0.smithi017.stdout:1236: expect (ObjNum 561 snap 131 seq_num 561) 2023-03-29T14:07:30.361 INFO:tasks.rados.rados.0.smithi017.stdout:1237: delete oid 36 current snap is 142 2023-03-29T14:07:30.384 INFO:tasks.rados.rados.0.smithi017.stdout:1237: done (2 left) 2023-03-29T14:07:30.385 INFO:tasks.rados.rados.0.smithi017.stdout:1238: snap_create 2023-03-29T14:07:30.392 INFO:tasks.rados.rados.0.smithi017.stdout:1234: finishing write tid 1 to smithi017107884-32 2023-03-29T14:07:30.395 INFO:tasks.rados.rados.0.smithi017.stdout:1236: expect (ObjNum 561 snap 131 seq_num 561) 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:1234: finishing write tid 2 to smithi017107884-32 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:1234: finishing write tid 3 to smithi017107884-32 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:1234: finishing write tid 4 to smithi017107884-32 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:1234: oid 32 updating version 0 to 257 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:1234: oid 32 version 257 is already newer than 257 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:1234: oid 32 version 257 is already newer than 257 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 257 (ObjNum 599 snap 142 seq_num 599) dirty exists 2023-03-29T14:07:30.630 INFO:tasks.rados.rados.0.smithi017.stdout:1234: left oid 32 (ObjNum 599 snap 142 seq_num 599) 2023-03-29T14:07:31.194 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1234: done (2 left) 2023-03-29T14:07:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1236: done (1 left) 2023-03-29T14:07:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1238: done (0 left) 2023-03-29T14:07:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1239: delete oid 15 current snap is 143 2023-03-29T14:07:31.324 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:31.322+0000 7f7eb312c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:31.328 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:31.327+0000 7f7eb312c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1239: done (0 left) 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1240: read oid 32 snap -1 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1240: expect (ObjNum 599 snap 142 seq_num 599) 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1241: read oid 49 snap -1 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1241: expect (ObjNum 496 snap 112 seq_num 496) 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1242: read oid 37 snap 137 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1242: expect deleted 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1243: write (excl) oid 28 current snap is 143 2023-03-29T14:07:31.396 INFO:tasks.rados.rados.0.smithi017.stdout:1243: seq_num 603 ranges {534621=567034,1636289=498332} 2023-03-29T14:07:31.397 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:31.397 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:31.397 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:31.402 INFO:tasks.rados.rados.0.smithi017.stdout:1243: writing smithi017107884-28 from 534621 to 1101655 tid 1 2023-03-29T14:07:31.408 INFO:tasks.rados.rados.0.smithi017.stdout:1243: writing smithi017107884-28 from 1636289 to 2134621 tid 2 2023-03-29T14:07:31.408 INFO:tasks.rados.rados.0.smithi017.stdout:1244: read oid 34 snap -1 2023-03-29T14:07:31.408 INFO:tasks.rados.rados.0.smithi017.stdout:1244: expect deleted 2023-03-29T14:07:31.409 INFO:tasks.rados.rados.0.smithi017.stdout:1245: write oid 40 current snap is 143 2023-03-29T14:07:31.409 INFO:tasks.rados.rados.0.smithi017.stdout:1245: seq_num 604 ranges {551428=642283,1694771=456657} 2023-03-29T14:07:31.415 INFO:tasks.rados.rados.0.smithi017.stdout:1245: writing smithi017107884-40 from 551428 to 1193711 tid 1 2023-03-29T14:07:31.421 INFO:tasks.rados.rados.0.smithi017.stdout:1245: writing smithi017107884-40 from 1694771 to 2151428 tid 2 2023-03-29T14:07:31.421 INFO:tasks.rados.rados.0.smithi017.stdout:1246: write (excl) oid 48 current snap is 143 2023-03-29T14:07:31.421 INFO:tasks.rados.rados.0.smithi017.stdout:1246: seq_num 605 ranges {568235=717532,1753253=414982} 2023-03-29T14:07:31.428 INFO:tasks.rados.rados.0.smithi017.stdout:1246: writing smithi017107884-48 from 568235 to 1285767 tid 1 2023-03-29T14:07:31.434 INFO:tasks.rados.rados.0.smithi017.stdout:1246: writing smithi017107884-48 from 1753253 to 2168235 tid 2 2023-03-29T14:07:31.434 INFO:tasks.rados.rados.0.smithi017.stdout:1247: snap_remove snap 142 2023-03-29T14:07:31.466 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:31.469 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:31.470 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:31.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:31.533+0000 7fb774b5b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:31.540+0000 7fb774b5b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:31.746 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:31.747 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:31.759 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2023-03-29T14:07:31.272690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225157286,"num_objects":668,"num_object_clones":572,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":34,"num_read":2753,"num_read_kb":1559738,"num_write":3836,"num_write_kb":1042458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2011,"ondisk_log_size":2011,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1331304,"kb_used_data":1261448,"kb_used_omap":0,"kb_used_meta":69824,"kb_avail":376156056,"statfs":{"total":386547056640,"available":385183801344,"internally_reserved":0,"allocated":1291722752,"data_stored":1568182138,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":71499776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.011641"},"pg_stats":[{"pgid":"3.f","version":"277'88","reported_seq":226,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:25.292875+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:25.292875+0000","last_peered":"2023-03-29T14:07:25.292875+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:25.292875+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12235892,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":41703,"num_write":98,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":311,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:24.423841+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:24.423841+0000","last_peered":"2023-03-29T14:07:24.423841+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:24.423841+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"300'191","reported_seq":669,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-03-29T14:07:26.506652+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:26.506652+0000","last_peered":"2023-03-29T14:07:26.506652+0000","last_clean":"2023-03-29T14:07:26.506652+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:26.506652+0000","last_undegraded":"2023-03-29T14:07:26.506652+0000","last_fullsized":"2023-03-29T14:07:26.506652+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36136040,"num_objects":64,"num_object_clones":54,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":257,"num_read_kb":168287,"num_write":368,"num_write_kb":110516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"297'173","reported_seq":640,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-03-29T14:07:25.268925+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:25.268925+0000","last_peered":"2023-03-29T14:07:25.268925+0000","last_clean":"2023-03-29T14:07:25.268925+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:25.268925+0000","last_undegraded":"2023-03-29T14:07:25.268925+0000","last_fullsized":"2023-03-29T14:07:25.268925+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20942116,"num_objects":70,"num_object_clones":62,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":262,"num_read_kb":158288,"num_write":340,"num_write_kb":95988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"298'181","reported_seq":624,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-03-29T14:07:25.323310+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:25.323310+0000","last_peered":"2023-03-29T14:07:25.323310+0000","last_clean":"2023-03-29T14:07:25.323310+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:25.323310+0000","last_undegraded":"2023-03-29T14:07:25.323310+0000","last_fullsized":"2023-03-29T14:07:25.323310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14420802,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":221,"num_read_kb":120997,"num_write":364,"num_write_kb":106102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":250,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:25.264314+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:25.264314+0000","last_peered":"2023-03-29T14:07:25.264314+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:25.264314+0000","last_undegraded":"2023-03-29T14:07:25.264314+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"260'104","reported_seq":473,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:26.293509+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:26.293509+0000","last_peered":"2023-03-29T14:07:26.293509+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:26.293509+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5748192,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":128,"num_read_kb":61239,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"300'75","reported_seq":414,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-03-29T14:07:26.382380+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:26.382380+0000","last_peered":"2023-03-29T14:07:26.382380+0000","last_clean":"2023-03-29T14:07:26.382380+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:26.382380+0000","last_undegraded":"2023-03-29T14:07:26.382380+0000","last_fullsized":"2023-03-29T14:07:26.382380+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":105,"num_read_kb":57966,"num_write":144,"num_write_kb":41374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:24.415460+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:24.415460+0000","last_peered":"2023-03-29T14:07:24.415460+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:24.415460+0000","last_undegraded":"2023-03-29T14:07:24.415460+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"297'67","reported_seq":157,"reported_epoch":298,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:23.318548+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:23.318548+0000","last_peered":"2023-03-29T14:07:23.318548+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:23.318548+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":30,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:25.274384+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:25.274384+0000","last_peered":"2023-03-29T14:07:25.274384+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:25.274384+0000","last_undegraded":"2023-03-29T14:07:25.274384+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":421,"reported_epoch":298,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:23.292235+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:23.292235+0000","last_peered":"2023-03-29T14:07:23.292235+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:23.292235+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":126,"num_read_kb":88536,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:24.375021+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:24.375021+0000","last_peered":"2023-03-29T14:07:24.375021+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:24.375021+0000","last_undegraded":"2023-03-29T14:07:24.375021+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"286'105","reported_seq":460,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:25.286605+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:25.286605+0000","last_peered":"2023-03-29T14:07:25.286605+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:25.286605+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13815750,"num_objects":38,"num_object_clones":32,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":58665,"num_write":208,"num_write_kb":61634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":253,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:25.276832+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:25.276832+0000","last_peered":"2023-03-29T14:07:25.276832+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:25.276832+0000","last_undegraded":"2023-03-29T14:07:25.276832+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":460,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:25.303694+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:25.303694+0000","last_peered":"2023-03-29T14:07:25.303694+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:25.303694+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"297'127","reported_seq":583,"reported_epoch":300,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:26.480422+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:26.480422+0000","last_peered":"2023-03-29T14:07:26.480422+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:26.480422+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19261416,"num_objects":44,"num_object_clones":36,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":269,"num_read_kb":186435,"num_write":252,"num_write_kb":73236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":44}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:25.268127+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:25.268127+0000","last_peered":"2023-03-29T14:07:25.268127+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:25.268127+0000","last_undegraded":"2023-03-29T14:07:25.268127+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"298'253","reported_seq":821,"reported_epoch":298,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:24.432696+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:24.432696+0000","last_peered":"2023-03-29T14:07:24.432696+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:24.432696+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40169918,"num_objects":86,"num_object_clones":72,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":401,"num_read_kb":247573,"num_write":478,"num_write_kb":138844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":86}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":234,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:24.400539+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:24.400539+0000","last_peered":"2023-03-29T14:07:24.400539+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:24.400539+0000","last_undegraded":"2023-03-29T14:07:24.400539+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"286'85","reported_seq":423,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-03-29T14:07:25.296364+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:25.296364+0000","last_peered":"2023-03-29T14:07:25.296364+0000","last_clean":"2023-03-29T14:07:25.296364+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:25.296364+0000","last_undegraded":"2023-03-29T14:07:25.296364+0000","last_fullsized":"2023-03-29T14:07:25.296364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7092752,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":123,"num_read_kb":70178,"num_write":176,"num_write_kb":50470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"282'184","reported_seq":625,"reported_epoch":300,"state":"active+clean","last_fresh":"2023-03-29T14:07:24.362418+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:24.362418+0000","last_peered":"2023-03-29T14:07:24.362418+0000","last_clean":"2023-03-29T14:07:24.362418+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:24.362418+0000","last_undegraded":"2023-03-29T14:07:24.362418+0000","last_fullsized":"2023-03-29T14:07:24.362418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":229,"num_read_kb":133906,"num_write":344,"num_write_kb":100514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":234,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:24.419913+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:24.419913+0000","last_peered":"2023-03-29T14:07:24.419913+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:24.419913+0000","last_undegraded":"2023-03-29T14:07:24.419913+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":300,"state":"active+undersized","last_fresh":"2023-03-29T14:07:25.272021+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:25.272021+0000","last_peered":"2023-03-29T14:07:25.272021+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:25.272021+0000","last_undegraded":"2023-03-29T14:07:25.272021+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"298'165","reported_seq":600,"reported_epoch":298,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:24.540012+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:24.540012+0000","last_peered":"2023-03-29T14:07:24.540012+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:24.540012+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20101766,"num_objects":58,"num_object_clones":50,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":214,"num_read_kb":127826,"num_write":320,"num_write_kb":96444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":58}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":223976550,"num_objects":664,"num_object_clones":572,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":34,"num_read":2611,"num_read_kb":1559618,"num_write":3562,"num_write_kb":1038670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1914,"ondisk_log_size":1914,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":371440,"kb_used_data":355496,"kb_used_omap":0,"kb_used_meta":15936,"kb_avail":94000400,"statfs":{"total":96636764160,"available":96256409600,"internally_reserved":0,"allocated":364027904,"data_stored":440943532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16318464},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":423160,"kb_used_data":401904,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":93948680,"statfs":{"total":96636764160,"available":96203448320,"internally_reserved":0,"allocated":411549696,"data_stored":504263017,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":380760,"kb_used_data":360208,"kb_used_omap":0,"kb_used_meta":20544,"kb_avail":93991080,"statfs":{"total":96636764160,"available":96246865920,"internally_reserved":0,"allocated":368852992,"data_stored":449743364,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21037056},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:32.278 INFO:tasks.rados.rados.0.smithi017.stdout:1247: done (7 left) 2023-03-29T14:07:32.278 INFO:tasks.rados.rados.0.smithi017.stdout:1248: read oid 15 snap -1 2023-03-29T14:07:32.279 INFO:tasks.rados.rados.0.smithi017.stdout:1248: expect deleted 2023-03-29T14:07:32.279 INFO:tasks.rados.rados.0.smithi017.stdout:1249: write oid 27 current snap is 143 2023-03-29T14:07:32.279 INFO:tasks.rados.rados.0.smithi017.stdout:1249: seq_num 606 ranges {585042=792781,2095382=89660} 2023-03-29T14:07:32.286 INFO:tasks.rados.rados.0.smithi017.stdout:1249: writing smithi017107884-27 from 585042 to 1377823 tid 1 2023-03-29T14:07:32.293 INFO:tasks.rados.rados.0.smithi017.stdout:1249: writing smithi017107884-27 from 2095382 to 2185042 tid 2 2023-03-29T14:07:32.293 INFO:tasks.rados.rados.0.smithi017.stdout:1250: read oid 21 snap -1 2023-03-29T14:07:32.293 INFO:tasks.rados.rados.0.smithi017.stdout:1250: expect (ObjNum 598 snap 141 seq_num 598) 2023-03-29T14:07:32.293 INFO:tasks.rados.rados.0.smithi017.stdout:1251: read oid 16 snap -1 2023-03-29T14:07:32.293 INFO:tasks.rados.rados.0.smithi017.stdout:1251: expect (ObjNum 525 snap 120 seq_num 525) 2023-03-29T14:07:32.293 INFO:tasks.rados.rados.0.smithi017.stdout:1252: snap_remove snap 141 2023-03-29T14:07:33.172 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~143s 2023-03-29T14:07:33.281 INFO:tasks.rados.rados.0.smithi017.stdout:1252: done (11 left) 2023-03-29T14:07:33.281 INFO:tasks.rados.rados.0.smithi017.stdout:1253: read oid 6 snap -1 2023-03-29T14:07:33.281 INFO:tasks.rados.rados.0.smithi017.stdout:1253: expect (ObjNum 581 snap 137 seq_num 581) 2023-03-29T14:07:33.281 INFO:tasks.rados.rados.0.smithi017.stdout:1254: snap_remove snap 138 2023-03-29T14:07:33.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:33.596+0000 7f89c6b7c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:33.601+0000 7f89c6b7c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:33.664 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:33.666 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:33.667 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:34.282 INFO:tasks.rados.rados.0.smithi017.stdout:1254: done (12 left) 2023-03-29T14:07:34.282 INFO:tasks.rados.rados.0.smithi017.stdout:1255: snap_remove snap 132 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1255: done (12 left) 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1256: read oid 17 snap -1 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1256: expect (ObjNum 583 snap 138 seq_num 583) 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1257: read oid 39 snap -1 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1257: expect (ObjNum 576 snap 136 seq_num 576) 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1258: write oid 10 current snap is 143 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1243: finishing write tid 1 to smithi017107884-28 2023-03-29T14:07:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1240: expect (ObjNum 599 snap 142 seq_num 599) 2023-03-29T14:07:35.421 INFO:tasks.rados.rados.0.smithi017.stdout:1241: expect (ObjNum 496 snap 112 seq_num 496) 2023-03-29T14:07:35.513 INFO:tasks.rados.rados.0.smithi017.stdout:1245: finishing write tid 1 to smithi017107884-40 2023-03-29T14:07:35.513 INFO:tasks.rados.rados.0.smithi017.stdout:1245: finishing write tid 2 to smithi017107884-40 2023-03-29T14:07:35.513 INFO:tasks.rados.rados.0.smithi017.stdout:1245: finishing write tid 3 to smithi017107884-40 2023-03-29T14:07:35.513 INFO:tasks.rados.rados.0.smithi017.stdout:1245: finishing write tid 4 to smithi017107884-40 2023-03-29T14:07:35.513 INFO:tasks.rados.rados.0.smithi017.stdout:1245: oid 40 updating version 0 to 177 2023-03-29T14:07:35.514 INFO:tasks.rados.rados.0.smithi017.stdout:1245: oid 40 version 177 is already newer than 177 2023-03-29T14:07:35.514 INFO:tasks.rados.rados.0.smithi017.stdout:1245: oid 40 version 177 is already newer than 177 2023-03-29T14:07:35.514 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 177 (ObjNum 604 snap 143 seq_num 604) dirty exists 2023-03-29T14:07:35.514 INFO:tasks.rados.rados.0.smithi017.stdout:1245: left oid 40 (ObjNum 604 snap 143 seq_num 604) 2023-03-29T14:07:35.514 INFO:tasks.rados.rados.0.smithi017.stdout:1243: finishing write tid 2 to smithi017107884-28 2023-03-29T14:07:35.514 INFO:tasks.rados.rados.0.smithi017.stdout:1258: seq_num 607 ranges {601849=468030,1753864=447985} 2023-03-29T14:07:35.519 INFO:tasks.rados.rados.0.smithi017.stdout:1258: writing smithi017107884-10 from 601849 to 1069879 tid 1 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1258: writing smithi017107884-10 from 1753864 to 2201849 tid 2 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1240: done (14 left) 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1241: done (13 left) 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1242: done (12 left) 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1245: done (11 left) 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1259: read oid 22 snap 137 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1259: expect (ObjNum 553 snap 127 seq_num 553) 2023-03-29T14:07:35.526 INFO:tasks.rados.rados.0.smithi017.stdout:1260: read oid 25 snap -1 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1260: expect (ObjNum 579 snap 137 seq_num 579) 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1261: delete oid 18 current snap is 143 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1243: finishing write tid 3 to smithi017107884-28 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1243: finishing write tid 4 to smithi017107884-28 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1243: oid 28 updating version 0 to 78 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1243: oid 28 version 78 is already newer than 78 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1243: oid 28 version 78 is already newer than 76 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 78 (ObjNum 603 snap 143 seq_num 603) dirty exists 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1243: left oid 28 (ObjNum 603 snap 143 seq_num 603) 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: finishing write tid 1 to smithi017107884-48 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: finishing write tid 2 to smithi017107884-48 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: finishing write tid 3 to smithi017107884-48 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: finishing write tid 4 to smithi017107884-48 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: oid 48 updating version 0 to 196 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: oid 48 version 196 is already newer than 196 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: oid 48 version 196 is already newer than 193 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 196 (ObjNum 605 snap 143 seq_num 605) dirty exists 2023-03-29T14:07:35.527 INFO:tasks.rados.rados.0.smithi017.stdout:1246: left oid 48 (ObjNum 605 snap 143 seq_num 605) 2023-03-29T14:07:35.528 INFO:tasks.rados.rados.0.smithi017.stdout:1251: expect (ObjNum 525 snap 120 seq_num 525) 2023-03-29T14:07:35.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:35.678 INFO:tasks.rados.rados.0.smithi017.stdout:1249: finishing write tid 1 to smithi017107884-27 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1249: finishing write tid 2 to smithi017107884-27 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1249: finishing write tid 3 to smithi017107884-27 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1243: done (13 left) 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1244: done (12 left) 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1246: done (11 left) 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1248: done (10 left) 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1251: done (9 left) 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1261: done (8 left) 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1262: read oid 31 snap -1 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1262: expect deleted 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1263: snap_create 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1249: finishing write tid 4 to smithi017107884-27 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1249: oid 27 updating version 0 to 135 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1249: oid 27 version 135 is already newer than 135 2023-03-29T14:07:35.679 INFO:tasks.rados.rados.0.smithi017.stdout:1249: oid 27 version 135 is already newer than 135 2023-03-29T14:07:35.680 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 135 (ObjNum 606 snap 143 seq_num 606) dirty exists 2023-03-29T14:07:35.680 INFO:tasks.rados.rados.0.smithi017.stdout:1249: left oid 27 (ObjNum 606 snap 143 seq_num 606) 2023-03-29T14:07:35.680 INFO:tasks.rados.rados.0.smithi017.stdout:1250: expect (ObjNum 598 snap 141 seq_num 598) 2023-03-29T14:07:35.777 INFO:tasks.rados.rados.0.smithi017.stdout:1253: expect (ObjNum 581 snap 137 seq_num 581) 2023-03-29T14:07:35.798 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:35.795+0000 7f5b4436a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:35.801 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:35.800+0000 7f5b4436a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:35.863 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:35.865 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:35.866 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:35.979 INFO:tasks.rados.rados.0.smithi017.stdout:1256: expect (ObjNum 583 snap 138 seq_num 583) 2023-03-29T14:07:36.126 INFO:tasks.rados.rados.0.smithi017.stdout:1257: expect (ObjNum 576 snap 136 seq_num 576) 2023-03-29T14:07:36.268 INFO:tasks.rados.rados.0.smithi017.stdout:1259: expect (ObjNum 553 snap 127 seq_num 553) 2023-03-29T14:07:36.508 INFO:tasks.rados.rados.0.smithi017.stdout:1260: expect (ObjNum 579 snap 137 seq_num 579) 2023-03-29T14:07:36.719 INFO:tasks.rados.rados.0.smithi017.stdout:1258: finishing write tid 1 to smithi017107884-10 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: finishing write tid 2 to smithi017107884-10 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: finishing write tid 3 to smithi017107884-10 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: finishing write tid 4 to smithi017107884-10 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: oid 10 updating version 0 to 200 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: oid 10 version 200 is already newer than 200 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: oid 10 version 200 is already newer than 200 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 200 (ObjNum 607 snap 143 seq_num 607) dirty exists 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: left oid 10 (ObjNum 607 snap 143 seq_num 607) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1249: done (9 left) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1250: done (8 left) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1253: done (7 left) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1256: done (6 left) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1257: done (5 left) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1258: done (4 left) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1259: done (3 left) 2023-03-29T14:07:36.720 INFO:tasks.rados.rados.0.smithi017.stdout:1260: done (2 left) 2023-03-29T14:07:36.721 INFO:tasks.rados.rados.0.smithi017.stdout:1262: done (1 left) 2023-03-29T14:07:36.721 INFO:tasks.rados.rados.0.smithi017.stdout:1263: done (0 left) 2023-03-29T14:07:36.721 INFO:tasks.rados.rados.0.smithi017.stdout:1264: write (excl) oid 40 current snap is 144 2023-03-29T14:07:36.721 INFO:tasks.rados.rados.0.smithi017.stdout:1264: seq_num 609 ranges {635463=734881,1987181=248282} 2023-03-29T14:07:36.728 INFO:tasks.rados.rados.0.smithi017.stdout:1264: writing smithi017107884-40 from 635463 to 1370344 tid 1 2023-03-29T14:07:36.733 INFO:tasks.rados.rados.0.smithi017.stdout:1264: writing smithi017107884-40 from 1987181 to 2235463 tid 2 2023-03-29T14:07:36.733 INFO:tasks.rados.rados.0.smithi017.stdout:1265: write (excl) oid 4 current snap is 144 2023-03-29T14:07:36.733 INFO:tasks.rados.rados.0.smithi017.stdout:1265: seq_num 610 ranges {652270=410130,1529310=493867} 2023-03-29T14:07:36.738 INFO:tasks.rados.rados.0.smithi017.stdout:1265: writing smithi017107884-4 from 652270 to 1062400 tid 1 2023-03-29T14:07:36.744 INFO:tasks.rados.rados.0.smithi017.stdout:1265: writing smithi017107884-4 from 1529310 to 2023177 tid 2 2023-03-29T14:07:36.744 INFO:tasks.rados.rados.0.smithi017.stdout:1266: write oid 30 current snap is 144 2023-03-29T14:07:36.744 INFO:tasks.rados.rados.0.smithi017.stdout:1266: seq_num 611 ranges {669077=485379,1587792=681285} 2023-03-29T14:07:36.750 INFO:tasks.rados.rados.0.smithi017.stdout:1266: writing smithi017107884-30 from 669077 to 1154456 tid 1 2023-03-29T14:07:36.757 INFO:tasks.rados.rados.0.smithi017.stdout:1266: writing smithi017107884-30 from 1587792 to 2269077 tid 2 2023-03-29T14:07:36.757 INFO:tasks.rados.rados.0.smithi017.stdout:1267: read oid 44 snap -1 2023-03-29T14:07:36.757 INFO:tasks.rados.rados.0.smithi017.stdout:1267: expect deleted 2023-03-29T14:07:36.757 INFO:tasks.rados.rados.0.smithi017.stdout:1268: write oid 41 current snap is 144 2023-03-29T14:07:36.757 INFO:tasks.rados.rados.0.smithi017.stdout:1268: seq_num 612 ranges {685884=560628,2046274=239610} 2023-03-29T14:07:36.763 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:36.764 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:36.764 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:36.764 INFO:tasks.rados.rados.0.smithi017.stdout:1268: writing smithi017107884-41 from 685884 to 1246512 tid 1 2023-03-29T14:07:36.769 INFO:tasks.rados.rados.0.smithi017.stdout:1268: writing smithi017107884-41 from 2046274 to 2285884 tid 2 2023-03-29T14:07:36.769 INFO:tasks.rados.rados.0.smithi017.stdout:1269: write oid 29 current snap is 144 2023-03-29T14:07:36.770 INFO:tasks.rados.rados.0.smithi017.stdout:1269: seq_num 613 ranges {702691=635877,2104756=197935} 2023-03-29T14:07:36.777 INFO:tasks.rados.rados.0.smithi017.stdout:1269: writing smithi017107884-29 from 702691 to 1338568 tid 1 2023-03-29T14:07:36.782 INFO:tasks.rados.rados.0.smithi017.stdout:1269: writing smithi017107884-29 from 2104756 to 2302691 tid 2 2023-03-29T14:07:36.782 INFO:tasks.rados.rados.0.smithi017.stdout:1270: write (excl) oid 1 current snap is 144 2023-03-29T14:07:36.782 INFO:tasks.rados.rados.0.smithi017.stdout:1270: seq_num 614 ranges {719498=711126,2046885=272613} 2023-03-29T14:07:36.790 INFO:tasks.rados.rados.0.smithi017.stdout:1270: writing smithi017107884-1 from 719498 to 1430624 tid 1 2023-03-29T14:07:36.796 INFO:tasks.rados.rados.0.smithi017.stdout:1270: writing smithi017107884-1 from 2046885 to 2319498 tid 2 2023-03-29T14:07:36.796 INFO:tasks.rados.rados.0.smithi017.stdout:1271: snap_remove snap 143 2023-03-29T14:07:36.910 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:36.905+0000 7f967fb01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:36.913 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:36.912+0000 7f967fb01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:37.124 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:37.126 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:37.138 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2023-03-29T14:07:36.290813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":202064468,"num_objects":690,"num_object_clones":594,"num_object_copies":2070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2814,"num_read_kb":1601277,"num_write":3908,"num_write_kb":1061962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2053,"ondisk_log_size":2053,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1359616,"kb_used_data":1289184,"kb_used_omap":0,"kb_used_meta":70400,"kb_avail":376127744,"statfs":{"total":386547056640,"available":385154809856,"internally_reserved":0,"allocated":1320124416,"data_stored":1603241207,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":72089600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6790226,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":14,"num_read_kb":10760,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014192"},"pg_stats":[{"pgid":"3.f","version":"303'92","reported_seq":238,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:30.288141+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:30.288141+0000","last_peered":"2023-03-29T14:07:30.288141+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:30.288141+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714908,"num_objects":14,"num_object_clones":14,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":41704,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":14}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":316,"reported_epoch":304,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:29.299580+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:29.299580+0000","last_peered":"2023-03-29T14:07:29.299580+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:29.299580+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"300'192","reported_seq":677,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-03-29T14:07:31.456665+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:31.456665+0000","last_peered":"2023-03-29T14:07:31.456665+0000","last_clean":"2023-03-29T14:07:31.456665+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:31.456665+0000","last_undegraded":"2023-03-29T14:07:31.456665+0000","last_fullsized":"2023-03-29T14:07:31.456665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36136040,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":168288,"num_write":370,"num_write_kb":111918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"305'176","reported_seq":651,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-03-29T14:07:31.440442+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:31.440442+0000","last_peered":"2023-03-29T14:07:31.440442+0000","last_clean":"2023-03-29T14:07:31.440442+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:31.440442+0000","last_undegraded":"2023-03-29T14:07:31.440442+0000","last_fullsized":"2023-03-29T14:07:31.440442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19026184,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":262,"num_read_kb":158288,"num_write":348,"num_write_kb":98136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"298'181","reported_seq":632,"reported_epoch":304,"state":"active+clean","last_fresh":"2023-03-29T14:07:30.387759+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:30.387759+0000","last_peered":"2023-03-29T14:07:30.387759+0000","last_clean":"2023-03-29T14:07:30.387759+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:30.387759+0000","last_undegraded":"2023-03-29T14:07:30.387759+0000","last_fullsized":"2023-03-29T14:07:30.387759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14420802,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":228,"num_read_kb":129858,"num_write":364,"num_write_kb":106102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":255,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-03-29T14:07:30.285626+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:30.285626+0000","last_peered":"2023-03-29T14:07:30.285626+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:30.285626+0000","last_undegraded":"2023-03-29T14:07:30.285626+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"301'107","reported_seq":483,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:30.275939+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:30.275939+0000","last_peered":"2023-03-29T14:07:30.275939+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:30.275939+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":135,"num_read_kb":67872,"num_write":190,"num_write_kb":54912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"305'77","reported_seq":425,"reported_epoch":305,"state":"active+clean","last_fresh":"2023-03-29T14:07:31.452206+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:31.452206+0000","last_peered":"2023-03-29T14:07:31.452206+0000","last_clean":"2023-03-29T14:07:31.452206+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:31.452206+0000","last_undegraded":"2023-03-29T14:07:31.452206+0000","last_fullsized":"2023-03-29T14:07:31.452206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12000660,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":57966,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":239,"reported_epoch":304,"state":"active+undersized","last_fresh":"2023-03-29T14:07:29.314882+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:29.314882+0000","last_peered":"2023-03-29T14:07:29.314882+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:29.314882+0000","last_undegraded":"2023-03-29T14:07:29.314882+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"297'67","reported_seq":165,"reported_epoch":308,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:33.308620+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:33.308620+0000","last_peered":"2023-03-29T14:07:33.308620+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:33.308620+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":30,"num_read_kb":12588,"num_write":52,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":258,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-03-29T14:07:30.290461+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:30.290461+0000","last_peered":"2023-03-29T14:07:30.290461+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:30.290461+0000","last_undegraded":"2023-03-29T14:07:30.290461+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"249'79","reported_seq":432,"reported_epoch":308,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:33.314342+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:33.314342+0000","last_peered":"2023-03-29T14:07:33.314342+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:33.314342+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420406,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":133,"num_read_kb":96673,"num_write":156,"num_write_kb":44944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":100,"reported_epoch":304,"state":"active+undersized","last_fresh":"2023-03-29T14:07:29.330391+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:29.330391+0000","last_peered":"2023-03-29T14:07:29.330391+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:29.330391+0000","last_undegraded":"2023-03-29T14:07:29.330391+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"300'107","reported_seq":470,"reported_epoch":304,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:30.336316+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:30.336316+0000","last_peered":"2023-03-29T14:07:30.336316+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:30.336316+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193594,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":134,"num_read_kb":65806,"num_write":208,"num_write_kb":61634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":258,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-03-29T14:07:30.287431+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:30.287431+0000","last_peered":"2023-03-29T14:07:30.287431+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:30.287431+0000","last_undegraded":"2023-03-29T14:07:30.287431+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":464,"reported_epoch":304,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:29.303633+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:29.303633+0000","last_peered":"2023-03-29T14:07:29.303633+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:29.303633+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"303'131","reported_seq":595,"reported_epoch":305,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:30.291230+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:30.291230+0000","last_peered":"2023-03-29T14:07:30.291230+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:30.291230+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18622816,"num_objects":46,"num_object_clones":38,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":270,"num_read_kb":186436,"num_write":260,"num_write_kb":75314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":46}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":258,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-03-29T14:07:30.289081+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:30.289081+0000","last_peered":"2023-03-29T14:07:30.289081+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:30.289081+0000","last_undegraded":"2023-03-29T14:07:30.289081+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"305'259","reported_seq":844,"reported_epoch":308,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:33.298551+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:33.298551+0000","last_peered":"2023-03-29T14:07:33.298551+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:33.298551+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30119200,"num_objects":90,"num_object_clones":76,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":416,"num_read_kb":258334,"num_write":486,"num_write_kb":140436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":90}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":239,"reported_epoch":304,"state":"active+undersized","last_fresh":"2023-03-29T14:07:29.336540+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:29.336540+0000","last_peered":"2023-03-29T14:07:29.336540+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:29.336540+0000","last_undegraded":"2023-03-29T14:07:29.336540+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":435,"reported_epoch":304,"state":"active+clean","last_fresh":"2023-03-29T14:07:29.316196+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:29.316196+0000","last_peered":"2023-03-29T14:07:29.316196+0000","last_clean":"2023-03-29T14:07:29.316196+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:29.316196+0000","last_undegraded":"2023-03-29T14:07:29.316196+0000","last_fullsized":"2023-03-29T14:07:29.316196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"282'184","reported_seq":631,"reported_epoch":304,"state":"active+clean","last_fresh":"2023-03-29T14:07:29.342507+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:29.342507+0000","last_peered":"2023-03-29T14:07:29.342507+0000","last_clean":"2023-03-29T14:07:29.342507+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:29.342507+0000","last_undegraded":"2023-03-29T14:07:29.342507+0000","last_fullsized":"2023-03-29T14:07:29.342507+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":230,"num_read_kb":133906,"num_write":344,"num_write_kb":100514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":239,"reported_epoch":304,"state":"active+undersized","last_fresh":"2023-03-29T14:07:29.328000+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:29.328000+0000","last_peered":"2023-03-29T14:07:29.328000+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:29.328000+0000","last_undegraded":"2023-03-29T14:07:29.328000+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":258,"reported_epoch":305,"state":"active+undersized","last_fresh":"2023-03-29T14:07:30.279509+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:30.279509+0000","last_peered":"2023-03-29T14:07:30.279509+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:30.279509+0000","last_undegraded":"2023-03-29T14:07:30.279509+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"303'177","reported_seq":630,"reported_epoch":308,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:33.294367+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:33.294367+0000","last_peered":"2023-03-29T14:07:33.294367+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:33.294367+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23328710,"num_objects":62,"num_object_clones":54,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":62,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":220,"num_read_kb":127828,"num_write":342,"num_write_kb":102746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":62}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":200883732,"num_objects":686,"num_object_clones":594,"num_object_copies":2058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2672,"num_read_kb":1601157,"num_write":3634,"num_write_kb":1058174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1956,"ondisk_log_size":1956,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":378072,"kb_used_data":362000,"kb_used_omap":0,"kb_used_meta":16064,"kb_avail":93993768,"statfs":{"total":96636764160,"available":96249618432,"internally_reserved":0,"allocated":370688000,"data_stored":449662017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16449536},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":436064,"kb_used_data":414552,"kb_used_omap":0,"kb_used_meta":21504,"kb_avail":93935776,"statfs":{"total":96636764160,"available":96190234624,"internally_reserved":0,"allocated":424501248,"data_stored":519992978,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22020096},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":389536,"kb_used_data":368792,"kb_used_omap":0,"kb_used_meta":20736,"kb_avail":93982304,"statfs":{"total":96636764160,"available":96237879296,"internally_reserved":0,"allocated":377643008,"data_stored":460353987,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21233664},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:37.295 INFO:tasks.rados.rados.0.smithi017.stdout:1271: done (7 left) 2023-03-29T14:07:37.296 INFO:tasks.rados.rados.0.smithi017.stdout:1272: write oid 26 current snap is 144 2023-03-29T14:07:37.296 INFO:tasks.rados.rados.0.smithi017.stdout:1266: finishing write tid 1 to smithi017107884-30 2023-03-29T14:07:37.296 INFO:tasks.rados.rados.0.smithi017.stdout:1272: seq_num 615 ranges {736305=786375,2105367=230938} 2023-03-29T14:07:37.304 INFO:tasks.rados.rados.0.smithi017.stdout:1272: writing smithi017107884-26 from 736305 to 1522680 tid 1 2023-03-29T14:07:37.308 INFO:tasks.rados.rados.0.smithi017.stdout:1272: writing smithi017107884-26 from 2105367 to 2336305 tid 2 2023-03-29T14:07:37.308 INFO:tasks.rados.rados.0.smithi017.stdout:1273: write (excl) oid 31 current snap is 144 2023-03-29T14:07:37.308 INFO:tasks.rados.rados.0.smithi017.stdout:1273: seq_num 616 ranges {753112=577977,1880202=472910} 2023-03-29T14:07:37.315 INFO:tasks.rados.rados.0.smithi017.stdout:1273: writing smithi017107884-31 from 753112 to 1331089 tid 1 2023-03-29T14:07:37.321 INFO:tasks.rados.rados.0.smithi017.stdout:1273: writing smithi017107884-31 from 1880202 to 2353112 tid 2 2023-03-29T14:07:37.322 INFO:tasks.rados.rados.0.smithi017.stdout:1274: read oid 39 snap -1 2023-03-29T14:07:37.322 INFO:tasks.rados.rados.0.smithi017.stdout:1274: expect (ObjNum 576 snap 136 seq_num 576) 2023-03-29T14:07:37.322 INFO:tasks.rados.rados.0.smithi017.stdout:1275: write oid 27 current snap is 144 2023-03-29T14:07:37.322 INFO:tasks.rados.rados.0.smithi017.stdout:1266: finishing write tid 2 to smithi017107884-30 2023-03-29T14:07:37.322 INFO:tasks.rados.rados.0.smithi017.stdout:1275: seq_num 617 ranges {769919=653226,1938684=431235} 2023-03-29T14:07:37.329 INFO:tasks.rados.rados.0.smithi017.stdout:1275: writing smithi017107884-27 from 769919 to 1423145 tid 1 2023-03-29T14:07:37.334 INFO:tasks.rados.rados.0.smithi017.stdout:1275: writing smithi017107884-27 from 1938684 to 2369919 tid 2 2023-03-29T14:07:37.334 INFO:tasks.rados.rados.0.smithi017.stdout:1276: snap_remove snap 139 2023-03-29T14:07:37.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:37.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:37.993+0000 7f66e2c9e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:37.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:37.998+0000 7f66e2c9e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:38.060 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:38.063 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:38.063 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1276: done (11 left) 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1277: delete oid 47 current snap is 144 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1264: finishing write tid 1 to smithi017107884-40 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1266: finishing write tid 3 to smithi017107884-30 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1266: finishing write tid 4 to smithi017107884-30 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1266: oid 30 updating version 0 to 69 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1266: oid 30 updating version 69 to 70 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1266: oid 30 version 70 is already newer than 70 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 70 (ObjNum 611 snap 144 seq_num 611) dirty exists 2023-03-29T14:07:38.238 INFO:tasks.rados.rados.0.smithi017.stdout:1266: left oid 30 (ObjNum 611 snap 144 seq_num 611) 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1265: finishing write tid 1 to smithi017107884-4 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1269: finishing write tid 1 to smithi017107884-29 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1269: finishing write tid 2 to smithi017107884-29 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1264: finishing write tid 2 to smithi017107884-40 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1264: finishing write tid 3 to smithi017107884-40 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1264: finishing write tid 4 to smithi017107884-40 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1264: oid 40 updating version 0 to 181 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1264: oid 40 version 181 is already newer than 181 2023-03-29T14:07:38.239 INFO:tasks.rados.rados.0.smithi017.stdout:1264: oid 40 version 181 is already newer than 178 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 181 (ObjNum 609 snap 144 seq_num 609) dirty exists 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1264: left oid 40 (ObjNum 609 snap 144 seq_num 609) 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1269: finishing write tid 3 to smithi017107884-29 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1268: finishing write tid 1 to smithi017107884-41 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1269: finishing write tid 4 to smithi017107884-29 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1269: oid 29 updating version 0 to 83 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1269: oid 29 version 83 is already newer than 83 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1269: oid 29 version 83 is already newer than 83 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 83 (ObjNum 613 snap 144 seq_num 613) dirty exists 2023-03-29T14:07:38.240 INFO:tasks.rados.rados.0.smithi017.stdout:1269: left oid 29 (ObjNum 613 snap 144 seq_num 613) 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1270: finishing write tid 1 to smithi017107884-1 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1268: finishing write tid 2 to smithi017107884-41 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1268: finishing write tid 3 to smithi017107884-41 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1268: finishing write tid 4 to smithi017107884-41 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1268: oid 41 updating version 0 to 188 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1268: oid 41 version 188 is already newer than 188 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1268: oid 41 version 188 is already newer than 188 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 188 (ObjNum 612 snap 144 seq_num 612) dirty exists 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1268: left oid 41 (ObjNum 612 snap 144 seq_num 612) 2023-03-29T14:07:38.241 INFO:tasks.rados.rados.0.smithi017.stdout:1265: finishing write tid 2 to smithi017107884-4 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1265: finishing write tid 3 to smithi017107884-4 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1265: finishing write tid 4 to smithi017107884-4 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1265: oid 4 updating version 0 to 204 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1265: oid 4 version 204 is already newer than 201 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1265: oid 4 version 204 is already newer than 204 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 204 (ObjNum 610 snap 144 seq_num 610) dirty exists 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1265: left oid 4 (ObjNum 610 snap 144 seq_num 610) 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1270: finishing write tid 2 to smithi017107884-1 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1270: finishing write tid 3 to smithi017107884-1 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1270: finishing write tid 4 to smithi017107884-1 2023-03-29T14:07:38.242 INFO:tasks.rados.rados.0.smithi017.stdout:1270: oid 1 updating version 0 to 180 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1270: oid 1 version 180 is already newer than 180 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1270: oid 1 version 180 is already newer than 178 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 180 (ObjNum 614 snap 144 seq_num 614) dirty exists 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1270: left oid 1 (ObjNum 614 snap 144 seq_num 614) 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1272: finishing write tid 1 to smithi017107884-26 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1272: finishing write tid 2 to smithi017107884-26 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1272: finishing write tid 3 to smithi017107884-26 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1272: finishing write tid 4 to smithi017107884-26 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1272: oid 26 updating version 0 to 138 2023-03-29T14:07:38.243 INFO:tasks.rados.rados.0.smithi017.stdout:1272: oid 26 version 138 is already newer than 138 2023-03-29T14:07:38.244 INFO:tasks.rados.rados.0.smithi017.stdout:1272: oid 26 version 138 is already newer than 138 2023-03-29T14:07:38.244 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 138 (ObjNum 615 snap 144 seq_num 615) dirty exists 2023-03-29T14:07:38.244 INFO:tasks.rados.rados.0.smithi017.stdout:1272: left oid 26 (ObjNum 615 snap 144 seq_num 615) 2023-03-29T14:07:38.244 INFO:tasks.rados.rados.0.smithi017.stdout:1273: finishing write tid 1 to smithi017107884-31 2023-03-29T14:07:38.244 INFO:tasks.rados.rados.0.smithi017.stdout:1274: expect (ObjNum 576 snap 136 seq_num 576) 2023-03-29T14:07:38.381 INFO:tasks.rados.rados.0.smithi017.stdout:1275: finishing write tid 1 to smithi017107884-27 2023-03-29T14:07:38.382 INFO:tasks.rados.rados.0.smithi017.stdout:1275: finishing write tid 2 to smithi017107884-27 2023-03-29T14:07:38.382 INFO:tasks.rados.rados.0.smithi017.stdout:1273: finishing write tid 2 to smithi017107884-31 2023-03-29T14:07:38.382 INFO:tasks.rados.rados.0.smithi017.stdout:1273: finishing write tid 3 to smithi017107884-31 2023-03-29T14:07:38.382 INFO:tasks.rados.rados.0.smithi017.stdout:1275: finishing write tid 3 to smithi017107884-27 2023-03-29T14:07:38.382 INFO:tasks.rados.rados.0.smithi017.stdout:1273: finishing write tid 4 to smithi017107884-31 2023-03-29T14:07:38.382 INFO:tasks.rados.rados.0.smithi017.stdout:1273: oid 31 updating version 0 to 111 2023-03-29T14:07:38.382 INFO:tasks.rados.rados.0.smithi017.stdout:1273: oid 31 version 111 is already newer than 111 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1273: oid 31 version 111 is already newer than 109 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 111 (ObjNum 616 snap 144 seq_num 616) dirty exists 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1273: left oid 31 (ObjNum 616 snap 144 seq_num 616) 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1275: finishing write tid 4 to smithi017107884-27 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1275: oid 27 updating version 0 to 141 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1275: oid 27 version 141 is already newer than 141 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1275: oid 27 version 141 is already newer than 141 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 141 (ObjNum 617 snap 144 seq_num 617) dirty exists 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1275: left oid 27 (ObjNum 617 snap 144 seq_num 617) 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1264: done (11 left) 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1265: done (10 left) 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1266: done (9 left) 2023-03-29T14:07:38.383 INFO:tasks.rados.rados.0.smithi017.stdout:1267: done (8 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1268: done (7 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1269: done (6 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1270: done (5 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1272: done (4 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1273: done (3 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1274: done (2 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1275: done (1 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1277: done (0 left) 2023-03-29T14:07:38.384 INFO:tasks.rados.rados.0.smithi017.stdout:1278: snap_create 2023-03-29T14:07:38.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~149s 2023-03-29T14:07:39.302 INFO:tasks.rados.rados.0.smithi017.stdout:1278: done (0 left) 2023-03-29T14:07:39.303 INFO:tasks.rados.rados.0.smithi017.stdout:1279: snap_remove snap 137 2023-03-29T14:07:40.064 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:40.191+0000 7f2f15b4d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:40.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:40.196+0000 7f2f15b4d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:40.259 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:40.262 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:40.262 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:40.307 INFO:tasks.rados.rados.0.smithi017.stdout:1279: done (0 left) 2023-03-29T14:07:40.307 INFO:tasks.rados.rados.0.smithi017.stdout:1280: write (excl) oid 43 current snap is 145 2023-03-29T14:07:40.307 INFO:tasks.rados.rados.0.smithi017.stdout:1280: seq_num 619 ranges {403533=403724,1539295=652201} 2023-03-29T14:07:40.311 INFO:tasks.rados.rados.0.smithi017.stdout:1280: writing smithi017107884-43 from 403533 to 807257 tid 1 2023-03-29T14:07:40.319 INFO:tasks.rados.rados.0.smithi017.stdout:1280: writing smithi017107884-43 from 1539295 to 2191496 tid 2 2023-03-29T14:07:40.319 INFO:tasks.rados.rados.0.smithi017.stdout:1281: snap_remove snap 144 2023-03-29T14:07:41.311 INFO:tasks.rados.rados.0.smithi017.stdout:1281: done (1 left) 2023-03-29T14:07:41.311 INFO:tasks.rados.rados.0.smithi017.stdout:1282: delete oid 28 current snap is 145 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:1280: finishing write tid 1 to smithi017107884-43 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:1280: finishing write tid 2 to smithi017107884-43 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:1280: finishing write tid 3 to smithi017107884-43 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:1280: finishing write tid 4 to smithi017107884-43 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:1280: oid 43 updating version 0 to 109 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:1280: oid 43 updating version 109 to 112 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:1280: oid 43 version 112 is already newer than 112 2023-03-29T14:07:41.312 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 112 (ObjNum 619 snap 145 seq_num 619) dirty exists 2023-03-29T14:07:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1280: left oid 43 (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:07:41.421 INFO:tasks.rados.rados.0.smithi017.stdout:1280: done (1 left) 2023-03-29T14:07:41.421 INFO:tasks.rados.rados.0.smithi017.stdout:1282: done (0 left) 2023-03-29T14:07:41.422 INFO:tasks.rados.rados.0.smithi017.stdout:1283: delete oid 45 current snap is 145 2023-03-29T14:07:41.469 INFO:tasks.rados.rados.0.smithi017.stdout:1283: done (0 left) 2023-03-29T14:07:41.470 INFO:tasks.rados.rados.0.smithi017.stdout:1284: read oid 27 snap -1 2023-03-29T14:07:41.470 INFO:tasks.rados.rados.0.smithi017.stdout:1284: expect (ObjNum 617 snap 144 seq_num 617) 2023-03-29T14:07:41.470 INFO:tasks.rados.rados.0.smithi017.stdout:1285: write (excl) oid 30 current snap is 145 2023-03-29T14:07:41.470 INFO:tasks.rados.rados.0.smithi017.stdout:1285: seq_num 622 ranges {453954=629471,1598388=515881} 2023-03-29T14:07:41.476 INFO:tasks.rados.rados.0.smithi017.stdout:1285: writing smithi017107884-30 from 453954 to 1083425 tid 1 2023-03-29T14:07:41.482 INFO:tasks.rados.rados.0.smithi017.stdout:1285: writing smithi017107884-30 from 1598388 to 2114269 tid 2 2023-03-29T14:07:41.482 INFO:tasks.rados.rados.0.smithi017.stdout:1286: delete oid 18 current snap is 145 2023-03-29T14:07:41.483 INFO:tasks.rados.rados.0.smithi017.stdout:1284: expect (ObjNum 617 snap 144 seq_num 617) 2023-03-29T14:07:41.627 INFO:tasks.rados.rados.0.smithi017.stdout:1285: finishing write tid 1 to smithi017107884-30 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: finishing write tid 2 to smithi017107884-30 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: finishing write tid 3 to smithi017107884-30 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: finishing write tid 4 to smithi017107884-30 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: oid 30 updating version 0 to 73 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: oid 30 version 73 is already newer than 71 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: oid 30 version 73 is already newer than 73 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 73 (ObjNum 622 snap 145 seq_num 622) dirty exists 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: left oid 30 (ObjNum 622 snap 145 seq_num 622) 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1284: done (2 left) 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1285: done (1 left) 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1286: done (0 left) 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1287: write (excl) oid 31 current snap is 145 2023-03-29T14:07:41.628 INFO:tasks.rados.rados.0.smithi017.stdout:1287: seq_num 624 ranges {487568=496322,1431705=519550} 2023-03-29T14:07:41.631 INFO:tasks.rados.rados.0.smithi017.stdout:1287: writing smithi017107884-31 from 487568 to 983890 tid 1 2023-03-29T14:07:41.636 INFO:tasks.rados.rados.0.smithi017.stdout:1287: writing smithi017107884-31 from 1431705 to 1951255 tid 2 2023-03-29T14:07:41.638 INFO:tasks.rados.rados.0.smithi017.stdout:1288: snap_create 2023-03-29T14:07:41.645 INFO:tasks.rados.rados.0.smithi017.stdout:1287: finishing write tid 1 to smithi017107884-31 2023-03-29T14:07:41.693 INFO:tasks.rados.rados.0.smithi017.stdout:1287: finishing write tid 2 to smithi017107884-31 2023-03-29T14:07:41.694 INFO:tasks.rados.rados.0.smithi017.stdout:1287: finishing write tid 3 to smithi017107884-31 2023-03-29T14:07:41.694 INFO:tasks.rados.rados.0.smithi017.stdout:1287: finishing write tid 4 to smithi017107884-31 2023-03-29T14:07:41.695 INFO:tasks.rados.rados.0.smithi017.stdout:1287: oid 31 updating version 0 to 115 2023-03-29T14:07:41.695 INFO:tasks.rados.rados.0.smithi017.stdout:1287: oid 31 version 115 is already newer than 115 2023-03-29T14:07:41.695 INFO:tasks.rados.rados.0.smithi017.stdout:1287: oid 31 version 115 is already newer than 113 2023-03-29T14:07:41.695 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 115 (ObjNum 624 snap 145 seq_num 624) dirty exists 2023-03-29T14:07:41.695 INFO:tasks.rados.rados.0.smithi017.stdout:1287: left oid 31 (ObjNum 624 snap 145 seq_num 624) 2023-03-29T14:07:42.143 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:42.143 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:42.143 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:42.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:42.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:42.270+0000 7f9e6e94f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:42.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:42.278+0000 7f9e6e94f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:42.314 INFO:tasks.rados.rados.0.smithi017.stdout:1287: done (1 left) 2023-03-29T14:07:42.315 INFO:tasks.rados.rados.0.smithi017.stdout:1288: done (0 left) 2023-03-29T14:07:42.315 INFO:tasks.rados.rados.0.smithi017.stdout:1289: delete oid 40 current snap is 146 2023-03-29T14:07:42.390 INFO:tasks.rados.rados.0.smithi017.stdout:1289: done (0 left) 2023-03-29T14:07:42.390 INFO:tasks.rados.rados.0.smithi017.stdout:1290: write (excl) oid 27 current snap is 146 2023-03-29T14:07:42.391 INFO:tasks.rados.rados.0.smithi017.stdout:1290: seq_num 626 ranges {521182=646820,1948669=523219} 2023-03-29T14:07:42.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:42.396+0000 7f91ca749700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:42.400 INFO:tasks.rados.rados.0.smithi017.stdout:1290: writing smithi017107884-27 from 521182 to 1168002 tid 1 2023-03-29T14:07:42.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:42.400+0000 7f91ca749700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:42.411 INFO:tasks.rados.rados.0.smithi017.stdout:1290: writing smithi017107884-27 from 1948669 to 2471888 tid 2 2023-03-29T14:07:42.411 INFO:tasks.rados.rados.0.smithi017.stdout:1291: delete oid 49 current snap is 146 2023-03-29T14:07:42.444 INFO:tasks.rados.rados.0.smithi017.stdout:1291: done (1 left) 2023-03-29T14:07:42.444 INFO:tasks.rados.rados.0.smithi017.stdout:1292: delete oid 21 current snap is 146 2023-03-29T14:07:42.461 INFO:tasks.rados.rados.0.smithi017.stdout:1290: finishing write tid 1 to smithi017107884-27 2023-03-29T14:07:42.482 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:42.485 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:42.485 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:42.500 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:42.503 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:42.512 INFO:tasks.rados.rados.0.smithi017.stdout:1290: finishing write tid 2 to smithi017107884-27 2023-03-29T14:07:42.513 INFO:tasks.rados.rados.0.smithi017.stdout:1290: finishing write tid 3 to smithi017107884-27 2023-03-29T14:07:42.514 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2023-03-29T14:07:42.313151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206704718,"num_objects":702,"num_object_clones":606,"num_object_copies":2106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":2905,"num_read_kb":1672981,"num_write":4010,"num_write_kb":1085608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2104,"ondisk_log_size":2104,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1392180,"kb_used_data":1321044,"kb_used_omap":0,"kb_used_meta":71104,"kb_avail":376095180,"statfs":{"total":386547056640,"available":385121464320,"internally_reserved":0,"allocated":1352749056,"data_stored":1643518014,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":72810496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11799834,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-8,"num_read":37,"num_read_kb":11264,"num_write":80,"num_write_kb":19316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.077893"},"pg_stats":[{"pgid":"3.f","version":"303'92","reported_seq":253,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:40.334920+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:40.334920+0000","last_peered":"2023-03-29T14:07:40.334920+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:40.334920+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714908,"num_objects":14,"num_object_clones":14,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":56805,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":14}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":323,"reported_epoch":314,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:38.306403+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:38.306403+0000","last_peered":"2023-03-29T14:07:38.306403+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:38.306403+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"310'204","reported_seq":714,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-03-29T14:07:40.330259+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:40.330259+0000","last_peered":"2023-03-29T14:07:40.330259+0000","last_clean":"2023-03-29T14:07:40.330259+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:40.330259+0000","last_undegraded":"2023-03-29T14:07:40.330259+0000","last_fullsized":"2023-03-29T14:07:40.330259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28203334,"num_objects":70,"num_object_clones":60,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":286,"num_read_kb":197311,"num_write":392,"num_write_kb":116290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"310'181","reported_seq":671,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-03-29T14:07:40.336396+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:40.336396+0000","last_peered":"2023-03-29T14:07:40.336396+0000","last_clean":"2023-03-29T14:07:40.336396+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:40.336396+0000","last_undegraded":"2023-03-29T14:07:40.336396+0000","last_fullsized":"2023-03-29T14:07:40.336396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19194254,"num_objects":74,"num_object_clones":66,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":273,"num_read_kb":168205,"num_write":356,"num_write_kb":100058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"298'181","reported_seq":639,"reported_epoch":314,"state":"active+clean","last_fresh":"2023-03-29T14:07:38.280552+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:38.280552+0000","last_peered":"2023-03-29T14:07:38.280552+0000","last_clean":"2023-03-29T14:07:38.280552+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:38.280552+0000","last_undegraded":"2023-03-29T14:07:38.280552+0000","last_fullsized":"2023-03-29T14:07:38.280552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14420802,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":228,"num_read_kb":129858,"num_write":364,"num_write_kb":106102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":264,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-03-29T14:07:40.329487+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:40.329487+0000","last_peered":"2023-03-29T14:07:40.329487+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:40.329487+0000","last_undegraded":"2023-03-29T14:07:40.329487+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"311'111","reported_seq":501,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:41.329913+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:41.329913+0000","last_peered":"2023-03-29T14:07:41.329913+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:41.329913+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706224,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":139,"num_read_kb":67873,"num_write":198,"num_write_kb":56966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"305'78","reported_seq":437,"reported_epoch":315,"state":"active+clean","last_fresh":"2023-03-29T14:07:41.340966+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:41.340966+0000","last_peered":"2023-03-29T14:07:41.340966+0000","last_clean":"2023-03-29T14:07:41.340966+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:41.340966+0000","last_undegraded":"2023-03-29T14:07:41.340966+0000","last_fullsized":"2023-03-29T14:07:41.340966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":246,"reported_epoch":314,"state":"active+undersized","last_fresh":"2023-03-29T14:07:38.300223+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:38.300223+0000","last_peered":"2023-03-29T14:07:38.300223+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:38.300223+0000","last_undegraded":"2023-03-29T14:07:38.300223+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"310'70","reported_seq":177,"reported_epoch":314,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:39.364438+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:39.364438+0000","last_peered":"2023-03-29T14:07:39.364438+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:39.364438+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4538154,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":31,"num_read_kb":12589,"num_write":60,"num_write_kb":15750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":267,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-03-29T14:07:40.334189+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:40.334189+0000","last_peered":"2023-03-29T14:07:40.334189+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:40.334189+0000","last_undegraded":"2023-03-29T14:07:40.334189+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"310'83","reported_seq":444,"reported_epoch":314,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:39.334357+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:39.334357+0000","last_peered":"2023-03-29T14:07:39.334357+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:39.334357+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":134,"num_read_kb":96674,"num_write":164,"num_write_kb":46574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":314,"state":"active+undersized","last_fresh":"2023-03-29T14:07:38.255278+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:38.255278+0000","last_peered":"2023-03-29T14:07:38.255278+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:38.255278+0000","last_undegraded":"2023-03-29T14:07:38.255278+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"314'112","reported_seq":484,"reported_epoch":314,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:40.465542+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:40.465542+0000","last_peered":"2023-03-29T14:07:40.465542+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:40.465542+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4807066,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":137,"num_read_kb":65807,"num_write":216,"num_write_kb":63698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":267,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-03-29T14:07:40.339041+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:40.339041+0000","last_peered":"2023-03-29T14:07:40.339041+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:40.339041+0000","last_undegraded":"2023-03-29T14:07:40.339041+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":471,"reported_epoch":314,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:38.292691+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:38.292691+0000","last_peered":"2023-03-29T14:07:38.292691+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:38.292691+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"311'141","reported_seq":634,"reported_epoch":315,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:41.479194+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:41.479194+0000","last_peered":"2023-03-29T14:07:41.479194+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:41.479194+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21043354,"num_objects":48,"num_object_clones":40,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":294,"num_read_kb":204092,"num_write":284,"num_write_kb":81148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":48}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":267,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-03-29T14:07:40.338228+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:40.338228+0000","last_peered":"2023-03-29T14:07:40.338228+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:40.338228+0000","last_undegraded":"2023-03-29T14:07:40.338228+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"305'259","reported_seq":849,"reported_epoch":314,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:39.327590+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:39.327590+0000","last_peered":"2023-03-29T14:07:39.327590+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:39.327590+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30119200,"num_objects":90,"num_object_clones":76,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":416,"num_read_kb":258334,"num_write":486,"num_write_kb":140436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":90}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":314,"state":"active+undersized","last_fresh":"2023-03-29T14:07:38.303215+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:38.303215+0000","last_peered":"2023-03-29T14:07:38.303215+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:38.303215+0000","last_undegraded":"2023-03-29T14:07:38.303215+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":442,"reported_epoch":314,"state":"active+clean","last_fresh":"2023-03-29T14:07:38.287659+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:38.287659+0000","last_peered":"2023-03-29T14:07:38.287659+0000","last_clean":"2023-03-29T14:07:38.287659+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:38.287659+0000","last_undegraded":"2023-03-29T14:07:38.287659+0000","last_fullsized":"2023-03-29T14:07:38.287659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"310'188","reported_seq":646,"reported_epoch":314,"state":"active+clean","last_fresh":"2023-03-29T14:07:38.310427+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:38.310427+0000","last_peered":"2023-03-29T14:07:38.310427+0000","last_clean":"2023-03-29T14:07:38.310427+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:38.310427+0000","last_undegraded":"2023-03-29T14:07:38.310427+0000","last_fullsized":"2023-03-29T14:07:38.310427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17950536,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":231,"num_read_kb":133907,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":246,"reported_epoch":314,"state":"active+undersized","last_fresh":"2023-03-29T14:07:38.272237+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:38.272237+0000","last_peered":"2023-03-29T14:07:38.272237+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:38.272237+0000","last_undegraded":"2023-03-29T14:07:38.272237+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":267,"reported_epoch":315,"state":"active+undersized","last_fresh":"2023-03-29T14:07:40.321120+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:40.321120+0000","last_peered":"2023-03-29T14:07:40.321120+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:40.321120+0000","last_undegraded":"2023-03-29T14:07:40.321120+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"310'180","reported_seq":642,"reported_epoch":314,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:39.342513+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:39.342513+0000","last_peered":"2023-03-29T14:07:39.342513+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:39.342513+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967706,"num_objects":62,"num_object_clones":54,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":62,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":127829,"num_write":350,"num_write_kb":104670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":62}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":205523982,"num_objects":698,"num_object_clones":606,"num_object_copies":2094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":2763,"num_read_kb":1672861,"num_write":3736,"num_write_kb":1081820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2007,"ondisk_log_size":2007,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":389624,"kb_used_data":373296,"kb_used_omap":0,"kb_used_meta":16320,"kb_avail":93982216,"statfs":{"total":96636764160,"available":96237789184,"internally_reserved":0,"allocated":382255104,"data_stored":463158504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16711680},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":443828,"kb_used_data":422124,"kb_used_omap":0,"kb_used_meta":21696,"kb_avail":93928012,"statfs":{"total":96636764160,"available":96182284288,"internally_reserved":0,"allocated":432254976,"data_stored":530265113,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22216704},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":402784,"kb_used_data":381784,"kb_used_omap":0,"kb_used_meta":20992,"kb_avail":93969056,"statfs":{"total":96636764160,"available":96224313344,"internally_reserved":0,"allocated":390946816,"data_stored":476862172,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21495808},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:42.518 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.5 1.0 2023-03-29T14:07:42.518 INFO:tasks.rados.rados.0.smithi017.stdout:1290: finishing write tid 4 to smithi017107884-27 2023-03-29T14:07:42.518 INFO:tasks.rados.rados.0.smithi017.stdout:1290: oid 27 updating version 0 to 145 2023-03-29T14:07:42.518 INFO:tasks.rados.rados.0.smithi017.stdout:1290: oid 27 version 145 is already newer than 142 2023-03-29T14:07:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1290: oid 27 version 145 is already newer than 145 2023-03-29T14:07:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 145 (ObjNum 626 snap 146 seq_num 626) dirty exists 2023-03-29T14:07:42.519 INFO:tasks.rados.rados.0.smithi017.stdout:1290: left oid 27 (ObjNum 626 snap 146 seq_num 626) 2023-03-29T14:07:42.560 INFO:tasks.rados.rados.0.smithi017.stdout:1290: done (1 left) 2023-03-29T14:07:42.560 INFO:tasks.rados.rados.0.smithi017.stdout:1292: done (0 left) 2023-03-29T14:07:42.560 INFO:tasks.rados.rados.0.smithi017.stdout:1293: snap_create 2023-03-29T14:07:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:42.646+0000 7f72a444a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:42.657 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:42.656+0000 7f72a444a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:42.860 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.5] on osd.0 to force-backfill; instructing pg(s) [1.0,3.e] on osd.1 to force-backfill; 2023-03-29T14:07:42.861 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:07:43.318 INFO:tasks.rados.rados.0.smithi017.stdout:1293: done (0 left) 2023-03-29T14:07:43.318 INFO:tasks.rados.rados.0.smithi017.stdout:1294: snap_create 2023-03-29T14:07:44.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~154s 2023-03-29T14:07:44.321 INFO:tasks.rados.rados.0.smithi017.stdout:1294: done (0 left) 2023-03-29T14:07:44.322 INFO:tasks.rados.rados.0.smithi017.stdout:1295: read oid 20 snap -1 2023-03-29T14:07:44.322 INFO:tasks.rados.rados.0.smithi017.stdout:1295: expect (ObjNum 561 snap 131 seq_num 561) 2023-03-29T14:07:44.322 INFO:tasks.rados.rados.0.smithi017.stdout:1296: snap_remove snap 146 2023-03-29T14:07:44.486 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:44.620 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:44.617+0000 7f6bbd949700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:44.623 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:44.622+0000 7f6bbd949700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:44.688 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:44.688 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:45.323 INFO:tasks.rados.rados.0.smithi017.stdout:1296: done (1 left) 2023-03-29T14:07:45.323 INFO:tasks.rados.rados.0.smithi017.stdout:1297: snap_create 2023-03-29T14:07:45.323 INFO:tasks.rados.rados.0.smithi017.stdout:1295: expect (ObjNum 561 snap 131 seq_num 561) 2023-03-29T14:07:46.329 INFO:tasks.rados.rados.0.smithi017.stdout:1295: done (1 left) 2023-03-29T14:07:46.330 INFO:tasks.rados.rados.0.smithi017.stdout:1297: done (0 left) 2023-03-29T14:07:46.330 INFO:tasks.rados.rados.0.smithi017.stdout:1298: delete oid 50 current snap is 149 2023-03-29T14:07:46.414 INFO:tasks.rados.rados.0.smithi017.stdout:1298: done (0 left) 2023-03-29T14:07:46.415 INFO:tasks.rados.rados.0.smithi017.stdout:1299: write (excl) oid 1 current snap is 149 2023-03-29T14:07:46.415 INFO:tasks.rados.rados.0.smithi017.stdout:1299: seq_num 630 ranges {588410=547816,1666244=414204,2484603=103807} 2023-03-29T14:07:46.421 INFO:tasks.rados.rados.0.smithi017.stdout:1299: writing smithi017107884-1 from 588410 to 1136226 tid 1 2023-03-29T14:07:46.426 INFO:tasks.rados.rados.0.smithi017.stdout:1299: writing smithi017107884-1 from 1666244 to 2080448 tid 2 2023-03-29T14:07:46.429 INFO:tasks.rados.rados.0.smithi017.stdout:1299: writing smithi017107884-1 from 2484603 to 2588410 tid 3 2023-03-29T14:07:46.429 INFO:tasks.rados.rados.0.smithi017.stdout:1300: snap_create 2023-03-29T14:07:46.452 INFO:tasks.rados.rados.0.smithi017.stdout:1299: finishing write tid 1 to smithi017107884-1 2023-03-29T14:07:46.482 INFO:tasks.rados.rados.0.smithi017.stdout:1299: finishing write tid 2 to smithi017107884-1 2023-03-29T14:07:46.511 INFO:tasks.rados.rados.0.smithi017.stdout:1299: finishing write tid 3 to smithi017107884-1 2023-03-29T14:07:46.518 INFO:tasks.rados.rados.0.smithi017.stdout:1299: finishing write tid 4 to smithi017107884-1 2023-03-29T14:07:46.520 INFO:tasks.rados.rados.0.smithi017.stdout:1299: finishing write tid 5 to smithi017107884-1 2023-03-29T14:07:46.520 INFO:tasks.rados.rados.0.smithi017.stdout:1299: oid 1 updating version 0 to 185 2023-03-29T14:07:46.520 INFO:tasks.rados.rados.0.smithi017.stdout:1299: oid 1 version 185 is already newer than 185 2023-03-29T14:07:46.520 INFO:tasks.rados.rados.0.smithi017.stdout:1299: oid 1 version 185 is already newer than 181 2023-03-29T14:07:46.520 INFO:tasks.rados.rados.0.smithi017.stdout:1299: oid 1 version 185 is already newer than 185 2023-03-29T14:07:46.521 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 185 (ObjNum 630 snap 149 seq_num 630) dirty exists 2023-03-29T14:07:46.521 INFO:tasks.rados.rados.0.smithi017.stdout:1299: left oid 1 (ObjNum 630 snap 149 seq_num 630) 2023-03-29T14:07:46.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:46.819+0000 7f4ff5b19700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:46.823+0000 7f4ff5b19700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:46.886 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:46.889 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:46.890 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:47.333 INFO:tasks.rados.rados.0.smithi017.stdout:1299: done (1 left) 2023-03-29T14:07:47.333 INFO:tasks.rados.rados.0.smithi017.stdout:1300: done (0 left) 2023-03-29T14:07:47.333 INFO:tasks.rados.rados.0.smithi017.stdout:1301: write (excl) oid 33 current snap is 150 2023-03-29T14:07:47.333 INFO:tasks.rados.rados.0.smithi017.stdout:1301: seq_num 631 ranges {605217=739418,2124726=480491} 2023-03-29T14:07:47.340 INFO:tasks.rados.rados.0.smithi017.stdout:1301: writing smithi017107884-33 from 605217 to 1344635 tid 1 2023-03-29T14:07:47.347 INFO:tasks.rados.rados.0.smithi017.stdout:1301: writing smithi017107884-33 from 2124726 to 2605217 tid 2 2023-03-29T14:07:47.347 INFO:tasks.rados.rados.0.smithi017.stdout:1302: snap_remove snap 145 2023-03-29T14:07:47.873 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:47.873 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:47.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:48.002+0000 7f1f7e4c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:48.009+0000 7f1f7e4c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:48.221 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:48.232 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2023-03-29T14:07:48.160600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186837920,"num_objects":710,"num_object_clones":614,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2929,"num_read_kb":1679473,"num_write":4034,"num_write_kb":1092118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2124,"ondisk_log_size":2124,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1388432,"kb_used_data":1317040,"kb_used_omap":0,"kb_used_meta":71360,"kb_avail":376098928,"statfs":{"total":386547056640,"available":385125302272,"internally_reserved":0,"allocated":1348648960,"data_stored":1638676822,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":73072640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19866798,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":24,"num_read_kb":6492,"num_write":24,"num_write_kb":6510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000729"},"pg_stats":[{"pgid":"3.f","version":"303'92","reported_seq":258,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:45.331391+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:45.331391+0000","last_peered":"2023-03-29T14:07:45.331391+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:45.331391+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9714908,"num_objects":14,"num_object_clones":14,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":56805,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":14}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":326,"reported_epoch":319,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:44.429235+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:44.429235+0000","last_peered":"2023-03-29T14:07:44.429235+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:44.429235+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"310'204","reported_seq":719,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-03-29T14:07:45.342373+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:45.342373+0000","last_peered":"2023-03-29T14:07:45.342373+0000","last_clean":"2023-03-29T14:07:45.342373+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:45.342373+0000","last_undegraded":"2023-03-29T14:07:45.342373+0000","last_fullsized":"2023-03-29T14:07:45.342373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28203334,"num_objects":70,"num_object_clones":60,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":286,"num_read_kb":197311,"num_write":392,"num_write_kb":116290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"316'183","reported_seq":678,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-03-29T14:07:45.344795+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:45.344795+0000","last_peered":"2023-03-29T14:07:45.344795+0000","last_clean":"2023-03-29T14:07:45.344795+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:45.344795+0000","last_undegraded":"2023-03-29T14:07:45.344795+0000","last_fullsized":"2023-03-29T14:07:45.344795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14723328,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":275,"num_read_kb":168205,"num_write":356,"num_write_kb":100058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"315'183","reported_seq":647,"reported_epoch":319,"state":"active+clean","last_fresh":"2023-03-29T14:07:44.476354+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:44.476354+0000","last_peered":"2023-03-29T14:07:44.476354+0000","last_clean":"2023-03-29T14:07:44.476354+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:44.476354+0000","last_undegraded":"2023-03-29T14:07:44.476354+0000","last_fullsized":"2023-03-29T14:07:44.476354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857454,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":237,"num_read_kb":136347,"num_write":364,"num_write_kb":106102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":269,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-03-29T14:07:45.341652+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:45.341652+0000","last_peered":"2023-03-29T14:07:45.341652+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:45.341652+0000","last_undegraded":"2023-03-29T14:07:45.341652+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":513,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:45.336216+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:45.336216+0000","last_peered":"2023-03-29T14:07:45.336216+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:45.336216+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":442,"reported_epoch":320,"state":"active+clean","last_fresh":"2023-03-29T14:07:45.345594+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:45.345594+0000","last_peered":"2023-03-29T14:07:45.345594+0000","last_clean":"2023-03-29T14:07:45.345594+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:45.345594+0000","last_undegraded":"2023-03-29T14:07:45.345594+0000","last_fullsized":"2023-03-29T14:07:45.345594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":249,"reported_epoch":319,"state":"active+undersized","last_fresh":"2023-03-29T14:07:44.416350+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:44.416350+0000","last_peered":"2023-03-29T14:07:44.416350+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:44.416350+0000","last_undegraded":"2023-03-29T14:07:44.416350+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"315'73","reported_seq":187,"reported_epoch":319,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:42.361851+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:42.361851+0000","last_peered":"2023-03-29T14:07:42.361851+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:42.361851+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4907908,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":34,"num_read_kb":12590,"num_write":68,"num_write_kb":17988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":12}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":272,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-03-29T14:07:45.337916+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:45.337916+0000","last_peered":"2023-03-29T14:07:45.337916+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:45.337916+0000","last_undegraded":"2023-03-29T14:07:45.337916+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"310'83","reported_seq":447,"reported_epoch":319,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:42.363881+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:42.363881+0000","last_peered":"2023-03-29T14:07:42.363881+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:42.363881+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":134,"num_read_kb":96674,"num_write":164,"num_write_kb":46574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":110,"reported_epoch":319,"state":"active+undersized","last_fresh":"2023-03-29T14:07:44.456435+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:44.456435+0000","last_peered":"2023-03-29T14:07:44.456435+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:44.456435+0000","last_undegraded":"2023-03-29T14:07:44.456435+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"314'112","reported_seq":487,"reported_epoch":319,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:44.426321+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:44.426321+0000","last_peered":"2023-03-29T14:07:44.426321+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:44.426321+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4807066,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":137,"num_read_kb":65807,"num_write":216,"num_write_kb":63698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":272,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-03-29T14:07:45.339667+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:45.339667+0000","last_peered":"2023-03-29T14:07:45.339667+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:45.339667+0000","last_undegraded":"2023-03-29T14:07:45.339667+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":474,"reported_epoch":319,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:44.448499+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:44.448499+0000","last_peered":"2023-03-29T14:07:44.448499+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:44.448499+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"316'147","reported_seq":648,"reported_epoch":320,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:45.343793+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:45.343793+0000","last_peered":"2023-03-29T14:07:45.343793+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:45.343793+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244708,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":299,"num_read_kb":204093,"num_write":292,"num_write_kb":83434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":272,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-03-29T14:07:45.340385+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:45.340385+0000","last_peered":"2023-03-29T14:07:45.340385+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:45.340385+0000","last_undegraded":"2023-03-29T14:07:45.340385+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"316'261","reported_seq":854,"reported_epoch":319,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:42.419583+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:42.419583+0000","last_peered":"2023-03-29T14:07:42.419583+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:42.419583+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25446656,"num_objects":92,"num_object_clones":78,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":92,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":418,"num_read_kb":258334,"num_write":486,"num_write_kb":140436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":92}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":249,"reported_epoch":319,"state":"active+undersized","last_fresh":"2023-03-29T14:07:44.402476+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:44.402476+0000","last_peered":"2023-03-29T14:07:44.402476+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:44.402476+0000","last_undegraded":"2023-03-29T14:07:44.402476+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":445,"reported_epoch":319,"state":"active+clean","last_fresh":"2023-03-29T14:07:44.396360+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:44.396360+0000","last_peered":"2023-03-29T14:07:44.396360+0000","last_clean":"2023-03-29T14:07:44.396360+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:44.396360+0000","last_undegraded":"2023-03-29T14:07:44.396360+0000","last_fullsized":"2023-03-29T14:07:44.396360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"310'188","reported_seq":649,"reported_epoch":319,"state":"active+clean","last_fresh":"2023-03-29T14:07:44.420355+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:44.420355+0000","last_peered":"2023-03-29T14:07:44.420355+0000","last_clean":"2023-03-29T14:07:44.420355+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:44.420355+0000","last_undegraded":"2023-03-29T14:07:44.420355+0000","last_fullsized":"2023-03-29T14:07:44.420355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17950536,"num_objects":66,"num_object_clones":56,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":231,"num_read_kb":133907,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":249,"reported_epoch":319,"state":"active+undersized","last_fresh":"2023-03-29T14:07:44.442557+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:44.442557+0000","last_peered":"2023-03-29T14:07:44.442557+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:44.442557+0000","last_undegraded":"2023-03-29T14:07:44.442557+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":272,"reported_epoch":320,"state":"active+undersized","last_fresh":"2023-03-29T14:07:45.332740+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:45.332740+0000","last_peered":"2023-03-29T14:07:45.332740+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:45.332740+0000","last_undegraded":"2023-03-29T14:07:45.332740+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"310'180","reported_seq":645,"reported_epoch":319,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:42.352058+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:42.352058+0000","last_peered":"2023-03-29T14:07:42.352058+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:42.352058+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967706,"num_objects":62,"num_object_clones":54,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":62,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":127829,"num_write":350,"num_write_kb":104670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":62}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":185657184,"num_objects":706,"num_object_clones":614,"num_object_copies":2118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2787,"num_read_kb":1679353,"num_write":3760,"num_write_kb":1088330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2027,"ondisk_log_size":2027,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":389096,"kb_used_data":372640,"kb_used_omap":0,"kb_used_meta":16448,"kb_avail":93982744,"statfs":{"total":96636764160,"available":96238329856,"internally_reserved":0,"allocated":381583360,"data_stored":462356281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":16842752},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":441404,"kb_used_data":419636,"kb_used_omap":0,"kb_used_meta":21760,"kb_avail":93930436,"statfs":{"total":96636764160,"available":96184766464,"internally_reserved":0,"allocated":429707264,"data_stored":527239257,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22282240},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":401988,"kb_used_data":380924,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":93969852,"statfs":{"total":96636764160,"available":96225128448,"internally_reserved":0,"allocated":390066176,"data_stored":475849059,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:48.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.9 1.0 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1302: done (1 left) 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1303: delete oid 41 current snap is 150 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: finishing write tid 1 to smithi017107884-33 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: finishing write tid 2 to smithi017107884-33 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: finishing write tid 3 to smithi017107884-33 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: finishing write tid 4 to smithi017107884-33 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: oid 33 updating version 0 to 189 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: oid 33 version 189 is already newer than 189 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: oid 33 version 189 is already newer than 186 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 189 (ObjNum 631 snap 150 seq_num 631) dirty exists 2023-03-29T14:07:48.241 INFO:tasks.rados.rados.0.smithi017.stdout:1301: left oid 33 (ObjNum 631 snap 150 seq_num 631) 2023-03-29T14:07:48.371 INFO:tasks.rados.rados.0.smithi017.stdout:1301: done (1 left) 2023-03-29T14:07:48.371 INFO:tasks.rados.rados.0.smithi017.stdout:1303: done (0 left) 2023-03-29T14:07:48.371 INFO:tasks.rados.rados.0.smithi017.stdout:1304: snap_create 2023-03-29T14:07:48.379 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:48.372+0000 7f8327eb3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:48.382+0000 7f8327eb3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [1.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.9] on osd.3 to force-recovery; 2023-03-29T14:07:48.587 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:07:48.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:49.019+0000 7f9ec65b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:49.023+0000 7f9ec65b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:49.091 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:49.091 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:49.341 INFO:tasks.rados.rados.0.smithi017.stdout:1304: done (0 left) 2023-03-29T14:07:49.341 INFO:tasks.rados.rados.0.smithi017.stdout:1305: delete oid 46 current snap is 151 2023-03-29T14:07:49.421 INFO:tasks.rados.rados.0.smithi017.stdout:1305: done (0 left) 2023-03-29T14:07:49.421 INFO:tasks.rados.rados.0.smithi017.stdout:1306: delete oid 16 current snap is 151 2023-03-29T14:07:49.445 INFO:tasks.rados.rados.0.smithi017.stdout:1306: done (0 left) 2023-03-29T14:07:49.445 INFO:tasks.rados.rados.0.smithi017.stdout:1307: write (excl) oid 24 current snap is 151 2023-03-29T14:07:49.445 INFO:tasks.rados.rados.0.smithi017.stdout:1307: seq_num 635 ranges {672445=640414,1842301=681553} 2023-03-29T14:07:49.452 INFO:tasks.rados.rados.0.smithi017.stdout:1307: writing smithi017107884-24 from 672445 to 1312859 tid 1 2023-03-29T14:07:49.459 INFO:tasks.rados.rados.0.smithi017.stdout:1307: writing smithi017107884-24 from 1842301 to 2523854 tid 2 2023-03-29T14:07:49.459 INFO:tasks.rados.rados.0.smithi017.stdout:1308: read oid 40 snap -1 2023-03-29T14:07:49.459 INFO:tasks.rados.rados.0.smithi017.stdout:1308: expect deleted 2023-03-29T14:07:49.460 INFO:tasks.rados.rados.0.smithi017.stdout:1309: write oid 30 current snap is 151 2023-03-29T14:07:49.460 INFO:tasks.rados.rados.0.smithi017.stdout:1309: seq_num 636 ranges {689252=715663,1900783=425211} 2023-03-29T14:07:49.467 INFO:tasks.rados.rados.0.smithi017.stdout:1309: writing smithi017107884-30 from 689252 to 1404915 tid 1 2023-03-29T14:07:49.472 INFO:tasks.rados.rados.0.smithi017.stdout:1309: writing smithi017107884-30 from 1900783 to 2325994 tid 2 2023-03-29T14:07:49.472 INFO:tasks.rados.rados.0.smithi017.stdout:1310: write (excl) oid 23 current snap is 151 2023-03-29T14:07:49.472 INFO:tasks.rados.rados.0.smithi017.stdout:1310: seq_num 637 ranges {706059=790912,1959265=685222} 2023-03-29T14:07:49.480 INFO:tasks.rados.rados.0.smithi017.stdout:1310: writing smithi017107884-23 from 706059 to 1496971 tid 1 2023-03-29T14:07:49.486 INFO:tasks.rados.rados.0.smithi017.stdout:1310: writing smithi017107884-23 from 1959265 to 2644487 tid 2 2023-03-29T14:07:49.487 INFO:tasks.rados.rados.0.smithi017.stdout:1311: snap_create 2023-03-29T14:07:49.555 INFO:tasks.rados.rados.0.smithi017.stdout:1307: finishing write tid 1 to smithi017107884-24 2023-03-29T14:07:49.596 INFO:tasks.rados.rados.0.smithi017.stdout:1309: finishing write tid 1 to smithi017107884-30 2023-03-29T14:07:49.600 INFO:tasks.rados.rados.0.smithi017.stdout:1307: finishing write tid 2 to smithi017107884-24 2023-03-29T14:07:49.605 INFO:tasks.rados.rados.0.smithi017.stdout:1309: finishing write tid 2 to smithi017107884-30 2023-03-29T14:07:49.608 INFO:tasks.rados.rados.0.smithi017.stdout:1307: finishing write tid 3 to smithi017107884-24 2023-03-29T14:07:49.619 INFO:tasks.rados.rados.0.smithi017.stdout:1309: finishing write tid 3 to smithi017107884-30 2023-03-29T14:07:49.619 INFO:tasks.rados.rados.0.smithi017.stdout:1307: finishing write tid 4 to smithi017107884-24 2023-03-29T14:07:49.620 INFO:tasks.rados.rados.0.smithi017.stdout:1307: oid 24 updating version 0 to 86 2023-03-29T14:07:49.620 INFO:tasks.rados.rados.0.smithi017.stdout:1307: oid 24 version 86 is already newer than 84 2023-03-29T14:07:49.620 INFO:tasks.rados.rados.0.smithi017.stdout:1307: oid 24 version 86 is already newer than 86 2023-03-29T14:07:49.620 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 86 (ObjNum 635 snap 151 seq_num 635) dirty exists 2023-03-29T14:07:49.620 INFO:tasks.rados.rados.0.smithi017.stdout:1307: left oid 24 (ObjNum 635 snap 151 seq_num 635) 2023-03-29T14:07:49.631 INFO:tasks.rados.rados.0.smithi017.stdout:1309: finishing write tid 4 to smithi017107884-30 2023-03-29T14:07:49.631 INFO:tasks.rados.rados.0.smithi017.stdout:1309: oid 30 updating version 0 to 77 2023-03-29T14:07:49.631 INFO:tasks.rados.rados.0.smithi017.stdout:1309: oid 30 version 77 is already newer than 77 2023-03-29T14:07:49.631 INFO:tasks.rados.rados.0.smithi017.stdout:1309: oid 30 version 77 is already newer than 76 2023-03-29T14:07:49.631 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 77 (ObjNum 636 snap 151 seq_num 636) dirty exists 2023-03-29T14:07:49.631 INFO:tasks.rados.rados.0.smithi017.stdout:1309: left oid 30 (ObjNum 636 snap 151 seq_num 636) 2023-03-29T14:07:49.671 INFO:tasks.rados.rados.0.smithi017.stdout:1310: finishing write tid 1 to smithi017107884-23 2023-03-29T14:07:49.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~160s 2023-03-29T14:07:49.693 INFO:tasks.rados.rados.0.smithi017.stdout:1310: finishing write tid 2 to smithi017107884-23 2023-03-29T14:07:49.694 INFO:tasks.rados.rados.0.smithi017.stdout:1310: finishing write tid 3 to smithi017107884-23 2023-03-29T14:07:49.697 INFO:tasks.rados.rados.0.smithi017.stdout:1310: finishing write tid 4 to smithi017107884-23 2023-03-29T14:07:49.697 INFO:tasks.rados.rados.0.smithi017.stdout:1310: oid 23 updating version 0 to 262 2023-03-29T14:07:49.697 INFO:tasks.rados.rados.0.smithi017.stdout:1310: oid 23 updating version 262 to 265 2023-03-29T14:07:49.697 INFO:tasks.rados.rados.0.smithi017.stdout:1310: oid 23 version 265 is already newer than 265 2023-03-29T14:07:49.698 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 265 (ObjNum 637 snap 151 seq_num 637) dirty exists 2023-03-29T14:07:49.698 INFO:tasks.rados.rados.0.smithi017.stdout:1310: left oid 23 (ObjNum 637 snap 151 seq_num 637) 2023-03-29T14:07:50.345 INFO:tasks.rados.rados.0.smithi017.stdout:1307: done (4 left) 2023-03-29T14:07:50.345 INFO:tasks.rados.rados.0.smithi017.stdout:1308: done (3 left) 2023-03-29T14:07:50.346 INFO:tasks.rados.rados.0.smithi017.stdout:1309: done (2 left) 2023-03-29T14:07:50.346 INFO:tasks.rados.rados.0.smithi017.stdout:1310: done (1 left) 2023-03-29T14:07:50.346 INFO:tasks.rados.rados.0.smithi017.stdout:1311: done (0 left) 2023-03-29T14:07:50.346 INFO:tasks.rados.rados.0.smithi017.stdout:1312: read oid 1 snap 148 2023-03-29T14:07:50.346 INFO:tasks.rados.rados.0.smithi017.stdout:1312: expect (ObjNum 614 snap 144 seq_num 614) 2023-03-29T14:07:50.346 INFO:tasks.rados.rados.0.smithi017.stdout:1313: snap_remove snap 147 2023-03-29T14:07:51.093 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:51.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:51.223+0000 7f4528518700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:51.228+0000 7f4528518700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:51.290 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:51.294 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:51.294 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:51.346 INFO:tasks.rados.rados.0.smithi017.stdout:1313: done (1 left) 2023-03-29T14:07:51.347 INFO:tasks.rados.rados.0.smithi017.stdout:1314: write oid 39 current snap is 152 2023-03-29T14:07:51.347 INFO:tasks.rados.rados.0.smithi017.stdout:1314: seq_num 638 ranges {722866=466161,1617747=428880,2587634=135232} 2023-03-29T14:07:51.353 INFO:tasks.rados.rados.0.smithi017.stdout:1314: writing smithi017107884-39 from 722866 to 1189027 tid 1 2023-03-29T14:07:51.358 INFO:tasks.rados.rados.0.smithi017.stdout:1314: writing smithi017107884-39 from 1617747 to 2046627 tid 2 2023-03-29T14:07:51.361 INFO:tasks.rados.rados.0.smithi017.stdout:1314: writing smithi017107884-39 from 2587634 to 2722866 tid 3 2023-03-29T14:07:51.362 INFO:tasks.rados.rados.0.smithi017.stdout:1315: delete oid 34 current snap is 152 2023-03-29T14:07:51.362 INFO:tasks.rados.rados.0.smithi017.stdout:1312: expect (ObjNum 614 snap 144 seq_num 614) 2023-03-29T14:07:51.453 INFO:tasks.rados.rados.0.smithi017.stdout:1312: done (2 left) 2023-03-29T14:07:51.453 INFO:tasks.rados.rados.0.smithi017.stdout:1315: done (1 left) 2023-03-29T14:07:51.453 INFO:tasks.rados.rados.0.smithi017.stdout:1316: write (excl) oid 10 current snap is 152 2023-03-29T14:07:51.453 INFO:tasks.rados.rados.0.smithi017.stdout:1316: seq_num 640 ranges {756480=733012,2134711=432549} 2023-03-29T14:07:51.461 INFO:tasks.rados.rados.0.smithi017.stdout:1316: writing smithi017107884-10 from 756480 to 1489492 tid 1 2023-03-29T14:07:51.467 INFO:tasks.rados.rados.0.smithi017.stdout:1316: writing smithi017107884-10 from 2134711 to 2567260 tid 2 2023-03-29T14:07:51.467 INFO:tasks.rados.rados.0.smithi017.stdout:1317: read oid 28 snap -1 2023-03-29T14:07:51.467 INFO:tasks.rados.rados.0.smithi017.stdout:1317: expect deleted 2023-03-29T14:07:51.467 INFO:tasks.rados.rados.0.smithi017.stdout:1318: write (excl) oid 3 current snap is 152 2023-03-29T14:07:51.467 INFO:tasks.rados.rados.0.smithi017.stdout:1314: finishing write tid 1 to smithi017107884-39 2023-03-29T14:07:51.467 INFO:tasks.rados.rados.0.smithi017.stdout:1318: seq_num 641 ranges {773287=408261,1793193=576207} 2023-03-29T14:07:51.473 INFO:tasks.rados.rados.0.smithi017.stdout:1318: writing smithi017107884-3 from 773287 to 1181548 tid 1 2023-03-29T14:07:51.480 INFO:tasks.rados.rados.0.smithi017.stdout:1318: writing smithi017107884-3 from 1793193 to 2369400 tid 2 2023-03-29T14:07:51.480 INFO:tasks.rados.rados.0.smithi017.stdout:1319: write oid 36 current snap is 152 2023-03-29T14:07:51.480 INFO:tasks.rados.rados.0.smithi017.stdout:1319: seq_num 642 ranges {790094=483510,1851675=436218} 2023-03-29T14:07:51.487 INFO:tasks.rados.rados.0.smithi017.stdout:1319: writing smithi017107884-36 from 790094 to 1273604 tid 1 2023-03-29T14:07:51.493 INFO:tasks.rados.rados.0.smithi017.stdout:1319: writing smithi017107884-36 from 1851675 to 2287893 tid 2 2023-03-29T14:07:51.493 INFO:tasks.rados.rados.0.smithi017.stdout:1320: read oid 33 snap -1 2023-03-29T14:07:51.493 INFO:tasks.rados.rados.0.smithi017.stdout:1320: expect (ObjNum 631 snap 150 seq_num 631) 2023-03-29T14:07:51.493 INFO:tasks.rados.rados.0.smithi017.stdout:1321: snap_remove snap 151 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1321: done (6 left) 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1322: snap_create 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1314: finishing write tid 2 to smithi017107884-39 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1314: finishing write tid 3 to smithi017107884-39 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1314: finishing write tid 4 to smithi017107884-39 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1314: finishing write tid 5 to smithi017107884-39 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1314: oid 39 updating version 0 to 209 2023-03-29T14:07:52.353 INFO:tasks.rados.rados.0.smithi017.stdout:1314: oid 39 version 209 is already newer than 209 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1314: oid 39 version 209 is already newer than 209 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1314: oid 39 version 209 is already newer than 209 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 209 (ObjNum 638 snap 152 seq_num 638) dirty exists 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1314: left oid 39 (ObjNum 638 snap 152 seq_num 638) 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1318: finishing write tid 1 to smithi017107884-3 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1319: finishing write tid 1 to smithi017107884-36 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1319: finishing write tid 2 to smithi017107884-36 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1318: finishing write tid 2 to smithi017107884-3 2023-03-29T14:07:52.354 INFO:tasks.rados.rados.0.smithi017.stdout:1320: expect (ObjNum 631 snap 150 seq_num 631) 2023-03-29T14:07:52.483 INFO:tasks.rados.rados.0.smithi017.stdout:1318: finishing write tid 3 to smithi017107884-3 2023-03-29T14:07:52.483 INFO:tasks.rados.rados.0.smithi017.stdout:1319: finishing write tid 3 to smithi017107884-36 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1318: finishing write tid 4 to smithi017107884-3 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1318: oid 3 updating version 0 to 269 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1318: oid 3 version 269 is already newer than 266 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1318: oid 3 version 269 is already newer than 269 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 269 (ObjNum 641 snap 152 seq_num 641) dirty exists 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1318: left oid 3 (ObjNum 641 snap 152 seq_num 641) 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1319: finishing write tid 4 to smithi017107884-36 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1319: oid 36 updating version 0 to 116 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1319: oid 36 version 116 is already newer than 116 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1319: oid 36 version 116 is already newer than 116 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 116 (ObjNum 642 snap 152 seq_num 642) dirty exists 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1319: left oid 36 (ObjNum 642 snap 152 seq_num 642) 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1316: finishing write tid 1 to smithi017107884-10 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1316: finishing write tid 2 to smithi017107884-10 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1316: finishing write tid 3 to smithi017107884-10 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1316: finishing write tid 4 to smithi017107884-10 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1316: oid 10 updating version 0 to 213 2023-03-29T14:07:52.484 INFO:tasks.rados.rados.0.smithi017.stdout:1316: oid 10 version 213 is already newer than 210 2023-03-29T14:07:52.485 INFO:tasks.rados.rados.0.smithi017.stdout:1316: oid 10 version 213 is already newer than 213 2023-03-29T14:07:52.485 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 213 (ObjNum 640 snap 152 seq_num 640) dirty exists 2023-03-29T14:07:52.485 INFO:tasks.rados.rados.0.smithi017.stdout:1316: left oid 10 (ObjNum 640 snap 152 seq_num 640) 2023-03-29T14:07:53.295 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:53.354 INFO:tasks.rados.rados.0.smithi017.stdout:1314: done (6 left) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1316: done (5 left) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1317: done (4 left) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1318: done (3 left) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1319: done (2 left) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1320: done (1 left) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1322: done (0 left) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1323: read oid 22 snap -1 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1323: expect (ObjNum 597 snap 141 seq_num 597) 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1324: read oid 50 snap 152 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1324: expect deleted 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1325: write oid 30 current snap is 153 2023-03-29T14:07:53.355 INFO:tasks.rados.rados.0.smithi017.stdout:1325: seq_num 643 ranges {406901=558759,1393804=579876,2608396=198505} 2023-03-29T14:07:53.363 INFO:tasks.rados.rados.0.smithi017.stdout:1325: writing smithi017107884-30 from 406901 to 965660 tid 1 2023-03-29T14:07:53.371 INFO:tasks.rados.rados.0.smithi017.stdout:1325: writing smithi017107884-30 from 1393804 to 1973680 tid 2 2023-03-29T14:07:53.378 INFO:tasks.rados.rados.0.smithi017.stdout:1325: writing smithi017107884-30 from 2608396 to 2806901 tid 3 2023-03-29T14:07:53.378 INFO:tasks.rados.rados.0.smithi017.stdout:1326: snap_remove snap 149 2023-03-29T14:07:53.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:53.435+0000 7f72b2bdc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:53.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:53.440+0000 7f72b2bdc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:53.554 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:53.557 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:53.558 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:53.599 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:07:53.599 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:07:53.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:53.727+0000 7f57ecfbe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:53.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:53.735+0000 7f57ecfbe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:54.361 INFO:tasks.rados.rados.0.smithi017.stdout:1326: done (3 left) 2023-03-29T14:07:54.361 INFO:tasks.rados.rados.0.smithi017.stdout:1327: read oid 43 snap -1 2023-03-29T14:07:54.361 INFO:tasks.rados.rados.0.smithi017.stdout:1327: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:07:54.361 INFO:tasks.rados.rados.0.smithi017.stdout:1328: snap_create 2023-03-29T14:07:54.361 INFO:tasks.rados.rados.0.smithi017.stdout:1323: expect (ObjNum 597 snap 141 seq_num 597) 2023-03-29T14:07:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:07:54.376 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:07:54.494 INFO:tasks.rados.rados.0.smithi017.stdout:1325: finishing write tid 1 to smithi017107884-30 2023-03-29T14:07:54.494 INFO:tasks.rados.rados.0.smithi017.stdout:1325: finishing write tid 2 to smithi017107884-30 2023-03-29T14:07:54.494 INFO:tasks.rados.rados.0.smithi017.stdout:1325: finishing write tid 3 to smithi017107884-30 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1325: finishing write tid 4 to smithi017107884-30 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1325: finishing write tid 5 to smithi017107884-30 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1325: oid 30 updating version 0 to 82 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1325: oid 30 version 82 is already newer than 82 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1325: oid 30 version 82 is already newer than 82 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1325: oid 30 version 82 is already newer than 81 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 82 (ObjNum 643 snap 153 seq_num 643) dirty exists 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1325: left oid 30 (ObjNum 643 snap 153 seq_num 643) 2023-03-29T14:07:54.495 INFO:tasks.rados.rados.0.smithi017.stdout:1327: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:07:54.512 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:54.505+0000 7fcfe53c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:54.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:54.514+0000 7fcfe53c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:55.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~165s 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1323: done (4 left) 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1324: done (3 left) 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1325: done (2 left) 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1327: done (1 left) 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1328: done (0 left) 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1329: read oid 45 snap -1 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1329: expect deleted 2023-03-29T14:07:55.366 INFO:tasks.rados.rados.0.smithi017.stdout:1330: delete oid 15 current snap is 154 2023-03-29T14:07:55.367 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:07:55.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:07:55.435 INFO:tasks.rados.rados.0.smithi017.stdout:1330: done (1 left) 2023-03-29T14:07:55.436 INFO:tasks.rados.rados.0.smithi017.stdout:1331: delete oid 40 current snap is 154 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1331: done (1 left) 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1332: read oid 10 snap -1 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1332: expect (ObjNum 640 snap 152 seq_num 640) 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1333: read oid 41 snap 152 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1333: expect deleted 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1334: read oid 13 snap 150 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1334: expect deleted 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1335: read oid 11 snap -1 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1335: expect deleted 2023-03-29T14:07:55.438 INFO:tasks.rados.rados.0.smithi017.stdout:1336: delete oid 28 current snap is 154 2023-03-29T14:07:55.449 INFO:tasks.rados.rados.0.smithi017.stdout:1336: done (5 left) 2023-03-29T14:07:55.449 INFO:tasks.rados.rados.0.smithi017.stdout:1337: delete oid 33 current snap is 154 2023-03-29T14:07:55.458 INFO:tasks.rados.rados.0.smithi017.stdout:1332: expect (ObjNum 640 snap 152 seq_num 640) 2023-03-29T14:07:55.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:55.548+0000 7fcf0548b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:55.558 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:55.558+0000 7fcf0548b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:55.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1329: done (5 left) 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1332: done (4 left) 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1333: done (3 left) 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1334: done (2 left) 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1335: done (1 left) 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1337: done (0 left) 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1338: read oid 48 snap -1 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1338: expect (ObjNum 605 snap 143 seq_num 605) 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1339: write (excl) oid 2 current snap is 154 2023-03-29T14:07:55.606 INFO:tasks.rados.rados.0.smithi017.stdout:1339: seq_num 648 ranges {490936=651357,1686214=447225,2861864=29072} 2023-03-29T14:07:55.616 INFO:tasks.rados.rados.0.smithi017.stdout:1339: writing smithi017107884-2 from 490936 to 1142293 tid 1 2023-03-29T14:07:55.624 INFO:tasks.rados.rados.0.smithi017.stdout:1339: writing smithi017107884-2 from 1686214 to 2133439 tid 2 2023-03-29T14:07:55.629 INFO:tasks.rados.rados.0.smithi017.stdout:1339: writing smithi017107884-2 from 2861864 to 2890936 tid 3 2023-03-29T14:07:55.629 INFO:tasks.rados.rados.0.smithi017.stdout:1340: snap_create 2023-03-29T14:07:55.629 INFO:tasks.rados.rados.0.smithi017.stdout:1338: expect (ObjNum 605 snap 143 seq_num 605) 2023-03-29T14:07:55.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:55.697+0000 7f7c81717700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:55.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:55.703+0000 7f7c81717700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:55.763 INFO:tasks.rados.rados.0.smithi017.stdout:1339: finishing write tid 1 to smithi017107884-2 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: finishing write tid 2 to smithi017107884-2 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: finishing write tid 3 to smithi017107884-2 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: finishing write tid 4 to smithi017107884-2 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: finishing write tid 5 to smithi017107884-2 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: oid 2 updating version 0 to 275 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: oid 2 version 275 is already newer than 275 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: oid 2 version 275 is already newer than 271 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: oid 2 version 275 is already newer than 275 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 275 (ObjNum 648 snap 154 seq_num 648) dirty exists 2023-03-29T14:07:55.764 INFO:tasks.rados.rados.0.smithi017.stdout:1339: left oid 2 (ObjNum 648 snap 154 seq_num 648) 2023-03-29T14:07:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:55.782 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:55.783 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:56.368 INFO:tasks.rados.rados.0.smithi017.stdout:1338: done (2 left) 2023-03-29T14:07:56.368 INFO:tasks.rados.rados.0.smithi017.stdout:1339: done (1 left) 2023-03-29T14:07:56.368 INFO:tasks.rados.rados.0.smithi017.stdout:1340: done (0 left) 2023-03-29T14:07:56.369 INFO:tasks.rados.rados.0.smithi017.stdout:1341: write oid 23 current snap is 155 2023-03-29T14:07:56.369 INFO:tasks.rados.rados.0.smithi017.stdout:1341: seq_num 649 ranges {507743=726606,1744696=590883,2772934=134809} 2023-03-29T14:07:56.371 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:07:56.375 INFO:tasks.rados.rados.0.smithi017.stdout:1341: writing smithi017107884-23 from 507743 to 1234349 tid 1 2023-03-29T14:07:56.381 INFO:tasks.rados.rados.0.smithi017.stdout:1341: writing smithi017107884-23 from 1744696 to 2335579 tid 2 2023-03-29T14:07:56.385 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:07:56.385 INFO:tasks.rados.rados.0.smithi017.stdout:1341: writing smithi017107884-23 from 2772934 to 2907743 tid 3 2023-03-29T14:07:56.385 INFO:tasks.rados.rados.0.smithi017.stdout:1342: snap_remove snap 153 2023-03-29T14:07:56.524 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:56.519+0000 7f365bc1d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:56.527+0000 7f365bc1d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:57.371 INFO:tasks.rados.rados.0.smithi017.stdout:1342: done (1 left) 2023-03-29T14:07:57.371 INFO:tasks.rados.rados.0.smithi017.stdout:1343: delete oid 21 current snap is 155 2023-03-29T14:07:57.371 INFO:tasks.rados.rados.0.smithi017.stdout:1341: finishing write tid 1 to smithi017107884-23 2023-03-29T14:07:57.371 INFO:tasks.rados.rados.0.smithi017.stdout:1341: finishing write tid 2 to smithi017107884-23 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: finishing write tid 3 to smithi017107884-23 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: finishing write tid 4 to smithi017107884-23 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: finishing write tid 5 to smithi017107884-23 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: oid 23 updating version 0 to 280 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: oid 23 version 280 is already newer than 280 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: oid 23 version 280 is already newer than 280 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: oid 23 version 280 is already newer than 280 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 280 (ObjNum 649 snap 155 seq_num 649) dirty exists 2023-03-29T14:07:57.372 INFO:tasks.rados.rados.0.smithi017.stdout:1341: left oid 23 (ObjNum 649 snap 155 seq_num 649) 2023-03-29T14:07:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:07:57.386 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:07:57.386 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:57.412 INFO:tasks.rados.rados.0.smithi017.stdout:1341: done (1 left) 2023-03-29T14:07:57.412 INFO:tasks.rados.rados.0.smithi017.stdout:1343: done (0 left) 2023-03-29T14:07:57.413 INFO:tasks.rados.rados.0.smithi017.stdout:1344: write oid 46 current snap is 155 2023-03-29T14:07:57.413 INFO:tasks.rados.rados.0.smithi017.stdout:1344: seq_num 651 ranges {541357=477104,1745307=594552} 2023-03-29T14:07:57.418 INFO:tasks.rados.rados.0.smithi017.stdout:1344: writing smithi017107884-46 from 541357 to 1018461 tid 1 2023-03-29T14:07:57.425 INFO:tasks.rados.rados.0.smithi017.stdout:1344: writing smithi017107884-46 from 1745307 to 2339859 tid 2 2023-03-29T14:07:57.426 INFO:tasks.rados.rados.0.smithi017.stdout:1345: write oid 25 current snap is 155 2023-03-29T14:07:57.426 INFO:tasks.rados.rados.0.smithi017.stdout:1345: seq_num 652 ranges {558164=552353,1803789=454563,2855203=102961} 2023-03-29T14:07:57.432 INFO:tasks.rados.rados.0.smithi017.stdout:1345: writing smithi017107884-25 from 558164 to 1110517 tid 1 2023-03-29T14:07:57.438 INFO:tasks.rados.rados.0.smithi017.stdout:1345: writing smithi017107884-25 from 1803789 to 2258352 tid 2 2023-03-29T14:07:57.442 INFO:tasks.rados.rados.0.smithi017.stdout:1345: writing smithi017107884-25 from 2855203 to 2958164 tid 3 2023-03-29T14:07:57.442 INFO:tasks.rados.rados.0.smithi017.stdout:1346: write (excl) oid 15 current snap is 155 2023-03-29T14:07:57.442 INFO:tasks.rados.rados.0.smithi017.stdout:1346: seq_num 653 ranges {574971=627602,1862271=598221} 2023-03-29T14:07:57.448 INFO:tasks.rados.rados.0.smithi017.stdout:1346: writing smithi017107884-15 from 574971 to 1202573 tid 1 2023-03-29T14:07:57.455 INFO:tasks.rados.rados.0.smithi017.stdout:1346: writing smithi017107884-15 from 1862271 to 2460492 tid 2 2023-03-29T14:07:57.455 INFO:tasks.rados.rados.0.smithi017.stdout:1347: snap_remove snap 148 2023-03-29T14:07:57.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:57.521+0000 7f320417d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:57.528+0000 7f320417d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:57.738 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:57.740 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:57.752 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2023-03-29T14:07:57.370290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175745432,"num_objects":734,"num_object_clones":638,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":3005,"num_read_kb":1711882,"num_write":4130,"num_write_kb":1117124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2177,"ondisk_log_size":2177,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1415476,"kb_used_data":1343508,"kb_used_omap":0,"kb_used_meta":71936,"kb_avail":376071884,"statfs":{"total":386547056640,"available":385097609216,"internally_reserved":0,"allocated":1375752192,"data_stored":1675838415,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":73662464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-941060,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":28215,"num_write":42,"num_write_kb":9388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017438"},"pg_stats":[{"pgid":"3.f","version":"323'94","reported_seq":269,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.382022+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:54.382022+0000","last_peered":"2023-03-29T14:07:54.382022+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:54.382022+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067558,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":60675,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":335,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.415217+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:54.415217+0000","last_peered":"2023-03-29T14:07:54.415217+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:54.415217+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"325'213","reported_seq":747,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.615157+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:55.615157+0000","last_peered":"2023-03-29T14:07:55.615157+0000","last_clean":"2023-03-29T14:07:55.615157+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:55.615157+0000","last_undegraded":"2023-03-29T14:07:55.615157+0000","last_fullsized":"2023-03-29T14:07:55.615157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27396664,"num_objects":74,"num_object_clones":64,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":210130,"num_write":410,"num_write_kb":120584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"316'183","reported_seq":685,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.371120+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:55.371120+0000","last_peered":"2023-03-29T14:07:55.371120+0000","last_clean":"2023-03-29T14:07:55.371120+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:55.371120+0000","last_undegraded":"2023-03-29T14:07:55.371120+0000","last_fullsized":"2023-03-29T14:07:55.371120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14723328,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":276,"num_read_kb":168205,"num_write":356,"num_write_kb":100058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"315'183","reported_seq":658,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.430570+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:54.430570+0000","last_peered":"2023-03-29T14:07:54.430570+0000","last_clean":"2023-03-29T14:07:54.430570+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:54.430570+0000","last_undegraded":"2023-03-29T14:07:54.430570+0000","last_fullsized":"2023-03-29T14:07:54.430570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857454,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":238,"num_read_kb":136347,"num_write":364,"num_write_kb":106102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":275,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.367754+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:54.367754+0000","last_peered":"2023-03-29T14:07:54.367754+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:54.367754+0000","last_undegraded":"2023-03-29T14:07:54.367754+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":519,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.366324+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:54.366324+0000","last_peered":"2023-03-29T14:07:54.366324+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:54.366324+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":449,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.369515+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:55.369515+0000","last_peered":"2023-03-29T14:07:55.369515+0000","last_clean":"2023-03-29T14:07:55.369515+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:55.369515+0000","last_undegraded":"2023-03-29T14:07:55.369515+0000","last_fullsized":"2023-03-29T14:07:55.369515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.421523+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:54.421523+0000","last_peered":"2023-03-29T14:07:54.421523+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:54.421523+0000","last_undegraded":"2023-03-29T14:07:54.421523+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"327'82","reported_seq":213,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.500773+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:54.500773+0000","last_peered":"2023-03-29T14:07:54.500773+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:54.500773+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5613802,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":12592,"num_write":86,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.379555+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:54.379555+0000","last_peered":"2023-03-29T14:07:54.379555+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:54.379555+0000","last_undegraded":"2023-03-29T14:07:54.379555+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"323'86","reported_seq":464,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.532504+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:54.532504+0000","last_peered":"2023-03-29T14:07:54.532504+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:54.532504+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9950272,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":96675,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.427478+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:54.427478+0000","last_peered":"2023-03-29T14:07:54.427478+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:54.427478+0000","last_undegraded":"2023-03-29T14:07:54.427478+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"325'116","reported_seq":507,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.460546+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:54.460546+0000","last_peered":"2023-03-29T14:07:54.460546+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:54.460546+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10387254,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":71578,"num_write":224,"num_write_kb":65496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.378856+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:54.378856+0000","last_peered":"2023-03-29T14:07:54.378856+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:54.378856+0000","last_undegraded":"2023-03-29T14:07:54.378856+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":483,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.433625+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:54.433625+0000","last_peered":"2023-03-29T14:07:54.433625+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:54.433625+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"316'147","reported_seq":654,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.382957+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:54.382957+0000","last_peered":"2023-03-29T14:07:54.382957+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:54.382957+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244708,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":299,"num_read_kb":204093,"num_write":292,"num_write_kb":83434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.378127+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:54.378127+0000","last_peered":"2023-03-29T14:07:54.378127+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:54.378127+0000","last_undegraded":"2023-03-29T14:07:54.378127+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"329'275","reported_seq":888,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:55.696460+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:55.696460+0000","last_peered":"2023-03-29T14:07:55.696460+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:55.696460+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20875350,"num_objects":98,"num_object_clones":84,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":427,"num_read_kb":258337,"num_write":512,"num_write_kb":147452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":98}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.410263+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:54.410263+0000","last_peered":"2023-03-29T14:07:54.410263+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:54.410263+0000","last_undegraded":"2023-03-29T14:07:54.410263+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":454,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.394571+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:54.394571+0000","last_peered":"2023-03-29T14:07:54.394571+0000","last_clean":"2023-03-29T14:07:54.394571+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:54.394571+0000","last_undegraded":"2023-03-29T14:07:54.394571+0000","last_fullsized":"2023-03-29T14:07:54.394571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"322'191","reported_seq":660,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.387317+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:54.387317+0000","last_peered":"2023-03-29T14:07:54.387317+0000","last_clean":"2023-03-29T14:07:54.387317+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:54.387317+0000","last_undegraded":"2023-03-29T14:07:54.387317+0000","last_fullsized":"2023-03-29T14:07:54.387317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":233,"num_read_kb":133907,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.416591+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:54.416591+0000","last_peered":"2023-03-29T14:07:54.416591+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:54.416591+0000","last_undegraded":"2023-03-29T14:07:54.416591+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.373758+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:54.373758+0000","last_peered":"2023-03-29T14:07:54.373758+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:54.373758+0000","last_undegraded":"2023-03-29T14:07:54.373758+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"321'189","reported_seq":679,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:55.456138+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:55.456138+0000","last_peered":"2023-03-29T14:07:55.456138+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:55.456138+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20841340,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":245,"num_read_kb":137772,"num_write":368,"num_write_kb":109140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":174564696,"num_objects":730,"num_object_clones":638,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":2863,"num_read_kb":1711762,"num_write":3856,"num_write_kb":1113336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2080,"ondisk_log_size":2080,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":397496,"kb_used_data":380848,"kb_used_omap":0,"kb_used_meta":16640,"kb_avail":93974344,"statfs":{"total":96636764160,"available":96229728256,"internally_reserved":0,"allocated":389988352,"data_stored":474771311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17039360},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":453152,"kb_used_data":431192,"kb_used_omap":0,"kb_used_meta":21952,"kb_avail":93918688,"statfs":{"total":96636764160,"available":96172736512,"internally_reserved":0,"allocated":441540608,"data_stored":543251323,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22478848},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":408884,"kb_used_data":387628,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":93962956,"statfs":{"total":96636764160,"available":96218066944,"internally_reserved":0,"allocated":396931072,"data_stored":484583556,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:57.757 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:57.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:07:57.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:57.928+0000 7f4e3e563700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:57.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:57.928+0000 7f99fbf68700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:57.934 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:57.934+0000 7f4e3e563700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:57.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:57.936+0000 7f99fbf68700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:58.039 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:07:58.041 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:07:58.042 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:07:58.242 INFO:tasks.rados.rados.0.smithi017.stdout:1347: done (3 left) 2023-03-29T14:07:58.242 INFO:tasks.rados.rados.0.smithi017.stdout:1348: snap_remove snap 154 2023-03-29T14:07:58.343 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:58.344 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:58.362 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2023-03-29T14:07:58.240667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175745432,"num_objects":734,"num_object_clones":638,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":3005,"num_read_kb":1711882,"num_write":4130,"num_write_kb":1117124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2177,"ondisk_log_size":2177,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1415476,"kb_used_data":1343508,"kb_used_omap":0,"kb_used_meta":71936,"kb_avail":376071884,"statfs":{"total":386547056640,"available":385097609216,"internally_reserved":0,"allocated":1375752192,"data_stored":1675838415,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":73662464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-941060,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":28215,"num_write":42,"num_write_kb":9388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.881213"},"pg_stats":[{"pgid":"3.f","version":"323'94","reported_seq":269,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.382022+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:54.382022+0000","last_peered":"2023-03-29T14:07:54.382022+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:54.382022+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067558,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":60675,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":335,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.415217+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:54.415217+0000","last_peered":"2023-03-29T14:07:54.415217+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:54.415217+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"325'213","reported_seq":747,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.615157+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:55.615157+0000","last_peered":"2023-03-29T14:07:55.615157+0000","last_clean":"2023-03-29T14:07:55.615157+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:55.615157+0000","last_undegraded":"2023-03-29T14:07:55.615157+0000","last_fullsized":"2023-03-29T14:07:55.615157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27396664,"num_objects":74,"num_object_clones":64,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":210130,"num_write":410,"num_write_kb":120584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"316'183","reported_seq":685,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.371120+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:55.371120+0000","last_peered":"2023-03-29T14:07:55.371120+0000","last_clean":"2023-03-29T14:07:55.371120+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:55.371120+0000","last_undegraded":"2023-03-29T14:07:55.371120+0000","last_fullsized":"2023-03-29T14:07:55.371120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14723328,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":276,"num_read_kb":168205,"num_write":356,"num_write_kb":100058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"315'183","reported_seq":658,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.430570+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:54.430570+0000","last_peered":"2023-03-29T14:07:54.430570+0000","last_clean":"2023-03-29T14:07:54.430570+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:54.430570+0000","last_undegraded":"2023-03-29T14:07:54.430570+0000","last_fullsized":"2023-03-29T14:07:54.430570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857454,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":238,"num_read_kb":136347,"num_write":364,"num_write_kb":106102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":275,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.367754+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:54.367754+0000","last_peered":"2023-03-29T14:07:54.367754+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:54.367754+0000","last_undegraded":"2023-03-29T14:07:54.367754+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":519,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.366324+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:54.366324+0000","last_peered":"2023-03-29T14:07:54.366324+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:54.366324+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":449,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.369515+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:55.369515+0000","last_peered":"2023-03-29T14:07:55.369515+0000","last_clean":"2023-03-29T14:07:55.369515+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:55.369515+0000","last_undegraded":"2023-03-29T14:07:55.369515+0000","last_fullsized":"2023-03-29T14:07:55.369515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.421523+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:54.421523+0000","last_peered":"2023-03-29T14:07:54.421523+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:54.421523+0000","last_undegraded":"2023-03-29T14:07:54.421523+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"327'82","reported_seq":213,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.500773+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:54.500773+0000","last_peered":"2023-03-29T14:07:54.500773+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:54.500773+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5613802,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":12592,"num_write":86,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.379555+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:54.379555+0000","last_peered":"2023-03-29T14:07:54.379555+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:54.379555+0000","last_undegraded":"2023-03-29T14:07:54.379555+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"323'86","reported_seq":464,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.532504+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:54.532504+0000","last_peered":"2023-03-29T14:07:54.532504+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:54.532504+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9950272,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":96675,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.427478+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:54.427478+0000","last_peered":"2023-03-29T14:07:54.427478+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:54.427478+0000","last_undegraded":"2023-03-29T14:07:54.427478+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"325'116","reported_seq":507,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.460546+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:54.460546+0000","last_peered":"2023-03-29T14:07:54.460546+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:54.460546+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10387254,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":71578,"num_write":224,"num_write_kb":65496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.378856+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:54.378856+0000","last_peered":"2023-03-29T14:07:54.378856+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:54.378856+0000","last_undegraded":"2023-03-29T14:07:54.378856+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":483,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.433625+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:54.433625+0000","last_peered":"2023-03-29T14:07:54.433625+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:54.433625+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"316'147","reported_seq":654,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.382957+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:54.382957+0000","last_peered":"2023-03-29T14:07:54.382957+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:54.382957+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244708,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":299,"num_read_kb":204093,"num_write":292,"num_write_kb":83434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.378127+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:54.378127+0000","last_peered":"2023-03-29T14:07:54.378127+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:54.378127+0000","last_undegraded":"2023-03-29T14:07:54.378127+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"329'275","reported_seq":888,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:55.696460+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:55.696460+0000","last_peered":"2023-03-29T14:07:55.696460+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:55.696460+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20875350,"num_objects":98,"num_object_clones":84,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":427,"num_read_kb":258337,"num_write":512,"num_write_kb":147452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":98}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.410263+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:54.410263+0000","last_peered":"2023-03-29T14:07:54.410263+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:54.410263+0000","last_undegraded":"2023-03-29T14:07:54.410263+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":454,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.394571+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:54.394571+0000","last_peered":"2023-03-29T14:07:54.394571+0000","last_clean":"2023-03-29T14:07:54.394571+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:54.394571+0000","last_undegraded":"2023-03-29T14:07:54.394571+0000","last_fullsized":"2023-03-29T14:07:54.394571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"322'191","reported_seq":660,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.387317+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:54.387317+0000","last_peered":"2023-03-29T14:07:54.387317+0000","last_clean":"2023-03-29T14:07:54.387317+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:54.387317+0000","last_undegraded":"2023-03-29T14:07:54.387317+0000","last_fullsized":"2023-03-29T14:07:54.387317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":233,"num_read_kb":133907,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.416591+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:54.416591+0000","last_peered":"2023-03-29T14:07:54.416591+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:54.416591+0000","last_undegraded":"2023-03-29T14:07:54.416591+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.373758+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:54.373758+0000","last_peered":"2023-03-29T14:07:54.373758+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:54.373758+0000","last_undegraded":"2023-03-29T14:07:54.373758+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"321'189","reported_seq":679,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:55.456138+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:55.456138+0000","last_peered":"2023-03-29T14:07:55.456138+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:55.456138+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20841340,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":245,"num_read_kb":137772,"num_write":368,"num_write_kb":109140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":174564696,"num_objects":730,"num_object_clones":638,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":2863,"num_read_kb":1711762,"num_write":3856,"num_write_kb":1113336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2080,"ondisk_log_size":2080,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":397496,"kb_used_data":380848,"kb_used_omap":0,"kb_used_meta":16640,"kb_avail":93974344,"statfs":{"total":96636764160,"available":96229728256,"internally_reserved":0,"allocated":389988352,"data_stored":474771311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17039360},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":453152,"kb_used_data":431192,"kb_used_omap":0,"kb_used_meta":21952,"kb_avail":93918688,"statfs":{"total":96636764160,"available":96172736512,"internally_reserved":0,"allocated":441540608,"data_stored":543251323,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22478848},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":408884,"kb_used_data":387628,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":93962956,"statfs":{"total":96636764160,"available":96218066944,"internally_reserved":0,"allocated":396931072,"data_stored":484583556,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:58.366 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:07:58.367 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:07:58.367 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:07:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:58.495+0000 7f5f226e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:58.502+0000 7f5f226e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:58.710 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:07:58.712 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:07:58.724 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2023-03-29T14:07:58.240667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175745432,"num_objects":734,"num_object_clones":638,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":3005,"num_read_kb":1711882,"num_write":4130,"num_write_kb":1117124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2177,"ondisk_log_size":2177,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1415476,"kb_used_data":1343508,"kb_used_omap":0,"kb_used_meta":71936,"kb_avail":376071884,"statfs":{"total":386547056640,"available":385097609216,"internally_reserved":0,"allocated":1375752192,"data_stored":1675838415,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":73662464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-941060,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":28215,"num_write":42,"num_write_kb":9388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.881213"},"pg_stats":[{"pgid":"3.f","version":"323'94","reported_seq":269,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.382022+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:07:54.382022+0000","last_peered":"2023-03-29T14:07:54.382022+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:07:54.382022+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067558,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":98,"num_read_kb":60675,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":335,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.415217+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:54.415217+0000","last_peered":"2023-03-29T14:07:54.415217+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:54.415217+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"325'213","reported_seq":747,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.615157+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:07:55.615157+0000","last_peered":"2023-03-29T14:07:55.615157+0000","last_clean":"2023-03-29T14:07:55.615157+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:07:55.615157+0000","last_undegraded":"2023-03-29T14:07:55.615157+0000","last_fullsized":"2023-03-29T14:07:55.615157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27396664,"num_objects":74,"num_object_clones":64,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":210130,"num_write":410,"num_write_kb":120584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"316'183","reported_seq":685,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.371120+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:07:55.371120+0000","last_peered":"2023-03-29T14:07:55.371120+0000","last_clean":"2023-03-29T14:07:55.371120+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:07:55.371120+0000","last_undegraded":"2023-03-29T14:07:55.371120+0000","last_fullsized":"2023-03-29T14:07:55.371120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14723328,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":276,"num_read_kb":168205,"num_write":356,"num_write_kb":100058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"315'183","reported_seq":658,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.430570+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:54.430570+0000","last_peered":"2023-03-29T14:07:54.430570+0000","last_clean":"2023-03-29T14:07:54.430570+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:54.430570+0000","last_undegraded":"2023-03-29T14:07:54.430570+0000","last_fullsized":"2023-03-29T14:07:54.430570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857454,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":238,"num_read_kb":136347,"num_write":364,"num_write_kb":106102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":275,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.367754+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:07:54.367754+0000","last_peered":"2023-03-29T14:07:54.367754+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:07:54.367754+0000","last_undegraded":"2023-03-29T14:07:54.367754+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":519,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.366324+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:07:54.366324+0000","last_peered":"2023-03-29T14:07:54.366324+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:07:54.366324+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":449,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:55.369515+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:07:55.369515+0000","last_peered":"2023-03-29T14:07:55.369515+0000","last_clean":"2023-03-29T14:07:55.369515+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:07:55.369515+0000","last_undegraded":"2023-03-29T14:07:55.369515+0000","last_fullsized":"2023-03-29T14:07:55.369515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.421523+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:54.421523+0000","last_peered":"2023-03-29T14:07:54.421523+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:54.421523+0000","last_undegraded":"2023-03-29T14:07:54.421523+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"327'82","reported_seq":213,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.500773+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:07:54.500773+0000","last_peered":"2023-03-29T14:07:54.500773+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:07:54.500773+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5613802,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":12592,"num_write":86,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.379555+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:07:54.379555+0000","last_peered":"2023-03-29T14:07:54.379555+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:07:54.379555+0000","last_undegraded":"2023-03-29T14:07:54.379555+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"323'86","reported_seq":464,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.532504+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:07:54.532504+0000","last_peered":"2023-03-29T14:07:54.532504+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:07:54.532504+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9950272,"num_objects":34,"num_object_clones":30,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":96675,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":34}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.427478+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:54.427478+0000","last_peered":"2023-03-29T14:07:54.427478+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:54.427478+0000","last_undegraded":"2023-03-29T14:07:54.427478+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"325'116","reported_seq":507,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.460546+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:54.460546+0000","last_peered":"2023-03-29T14:07:54.460546+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:54.460546+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10387254,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":71578,"num_write":224,"num_write_kb":65496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.378856+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:07:54.378856+0000","last_peered":"2023-03-29T14:07:54.378856+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:07:54.378856+0000","last_undegraded":"2023-03-29T14:07:54.378856+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":483,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.433625+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:54.433625+0000","last_peered":"2023-03-29T14:07:54.433625+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:54.433625+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"316'147","reported_seq":654,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:54.382957+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:07:54.382957+0000","last_peered":"2023-03-29T14:07:54.382957+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:07:54.382957+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244708,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":299,"num_read_kb":204093,"num_write":292,"num_write_kb":83434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.378127+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:07:54.378127+0000","last_peered":"2023-03-29T14:07:54.378127+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:07:54.378127+0000","last_undegraded":"2023-03-29T14:07:54.378127+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"329'275","reported_seq":888,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:55.696460+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:07:55.696460+0000","last_peered":"2023-03-29T14:07:55.696460+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:07:55.696460+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20875350,"num_objects":98,"num_object_clones":84,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":427,"num_read_kb":258337,"num_write":512,"num_write_kb":147452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":98}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.410263+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:54.410263+0000","last_peered":"2023-03-29T14:07:54.410263+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:54.410263+0000","last_undegraded":"2023-03-29T14:07:54.410263+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":454,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.394571+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:54.394571+0000","last_peered":"2023-03-29T14:07:54.394571+0000","last_clean":"2023-03-29T14:07:54.394571+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:54.394571+0000","last_undegraded":"2023-03-29T14:07:54.394571+0000","last_fullsized":"2023-03-29T14:07:54.394571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"322'191","reported_seq":660,"reported_epoch":329,"state":"active+clean","last_fresh":"2023-03-29T14:07:54.387317+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:07:54.387317+0000","last_peered":"2023-03-29T14:07:54.387317+0000","last_clean":"2023-03-29T14:07:54.387317+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:07:54.387317+0000","last_undegraded":"2023-03-29T14:07:54.387317+0000","last_fullsized":"2023-03-29T14:07:54.387317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":233,"num_read_kb":133907,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":258,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.416591+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:54.416591+0000","last_peered":"2023-03-29T14:07:54.416591+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:54.416591+0000","last_undegraded":"2023-03-29T14:07:54.416591+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":329,"state":"active+undersized","last_fresh":"2023-03-29T14:07:54.373758+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:07:54.373758+0000","last_peered":"2023-03-29T14:07:54.373758+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:07:54.373758+0000","last_undegraded":"2023-03-29T14:07:54.373758+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"321'189","reported_seq":679,"reported_epoch":329,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:55.456138+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:07:55.456138+0000","last_peered":"2023-03-29T14:07:55.456138+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:07:55.456138+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20841340,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":245,"num_read_kb":137772,"num_write":368,"num_write_kb":109140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":174564696,"num_objects":730,"num_object_clones":638,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":2863,"num_read_kb":1711762,"num_write":3856,"num_write_kb":1113336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2080,"ondisk_log_size":2080,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":397496,"kb_used_data":380848,"kb_used_omap":0,"kb_used_meta":16640,"kb_avail":93974344,"statfs":{"total":96636764160,"available":96229728256,"internally_reserved":0,"allocated":389988352,"data_stored":474771311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17039360},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":453152,"kb_used_data":431192,"kb_used_omap":0,"kb_used_meta":21952,"kb_avail":93918688,"statfs":{"total":96636764160,"available":96172736512,"internally_reserved":0,"allocated":441540608,"data_stored":543251323,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22478848},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":408884,"kb_used_data":387628,"kb_used_omap":0,"kb_used_meta":21248,"kb_avail":93962956,"statfs":{"total":96636764160,"available":96218066944,"internally_reserved":0,"allocated":396931072,"data_stored":484583556,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21757952},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:07:58.728 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.0 3.6 2023-03-29T14:07:58.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:58.855+0000 7f15cc9e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:58.865 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:07:58.865+0000 7f15cc9e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:07:59.069 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.6] on osd.0 to force-backfill; instructing pg(s) [3.0,3.e] on osd.1 to force-backfill; 2023-03-29T14:07:59.069 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:07:59.246 INFO:tasks.rados.rados.0.smithi017.stdout:1348: done (3 left) 2023-03-29T14:07:59.246 INFO:tasks.rados.rados.0.smithi017.stdout:1349: snap_remove snap 150 2023-03-29T14:08:00.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:00.172 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:00.171+0000 7efee65dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:00.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:00.176+0000 7efee65dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:00.247 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:00.248 INFO:tasks.rados.rados.0.smithi017.stdout:1349: done (3 left) 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1350: delete oid 24 current snap is 155 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: finishing write tid 1 to smithi017107884-25 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: finishing write tid 2 to smithi017107884-25 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: finishing write tid 3 to smithi017107884-25 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: finishing write tid 4 to smithi017107884-25 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: finishing write tid 5 to smithi017107884-25 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: oid 25 updating version 0 to 189 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: oid 25 version 189 is already newer than 189 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: oid 25 version 189 is already newer than 189 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: oid 25 version 189 is already newer than 189 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 189 (ObjNum 652 snap 155 seq_num 652) dirty exists 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1345: left oid 25 (ObjNum 652 snap 155 seq_num 652) 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1344: finishing write tid 1 to smithi017107884-46 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1346: finishing write tid 1 to smithi017107884-15 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1344: finishing write tid 2 to smithi017107884-46 2023-03-29T14:08:00.249 INFO:tasks.rados.rados.0.smithi017.stdout:1344: finishing write tid 3 to smithi017107884-46 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1344: finishing write tid 4 to smithi017107884-46 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1344: oid 46 updating version 0 to 188 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1344: oid 46 version 188 is already newer than 188 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1344: oid 46 version 188 is already newer than 188 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 188 (ObjNum 651 snap 155 seq_num 651) dirty exists 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1344: left oid 46 (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1346: finishing write tid 2 to smithi017107884-15 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1346: finishing write tid 3 to smithi017107884-15 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1346: finishing write tid 4 to smithi017107884-15 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1346: oid 15 updating version 0 to 283 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1346: oid 15 version 283 is already newer than 283 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1346: oid 15 version 283 is already newer than 281 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 283 (ObjNum 653 snap 155 seq_num 653) dirty exists 2023-03-29T14:08:00.250 INFO:tasks.rados.rados.0.smithi017.stdout:1346: left oid 15 (ObjNum 653 snap 155 seq_num 653) 2023-03-29T14:08:00.251 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:00.251 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1344: done (3 left) 2023-03-29T14:08:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1345: done (2 left) 2023-03-29T14:08:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1346: done (1 left) 2023-03-29T14:08:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1350: done (0 left) 2023-03-29T14:08:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1351: read oid 29 snap 152 2023-03-29T14:08:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1351: expect (ObjNum 613 snap 144 seq_num 613) 2023-03-29T14:08:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1352: read oid 27 snap -1 2023-03-29T14:08:00.357 INFO:tasks.rados.rados.0.smithi017.stdout:1352: expect (ObjNum 626 snap 146 seq_num 626) 2023-03-29T14:08:00.357 INFO:tasks.rados.rados.0.smithi017.stdout:1353: read oid 12 snap -1 2023-03-29T14:08:00.357 INFO:tasks.rados.rados.0.smithi017.stdout:1353: expect (ObjNum 557 snap 131 seq_num 557) 2023-03-29T14:08:00.357 INFO:tasks.rados.rados.0.smithi017.stdout:1354: read oid 8 snap -1 2023-03-29T14:08:00.357 INFO:tasks.rados.rados.0.smithi017.stdout:1354: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:08:00.357 INFO:tasks.rados.rados.0.smithi017.stdout:1355: delete oid 23 current snap is 155 2023-03-29T14:08:00.375 INFO:tasks.rados.rados.0.smithi017.stdout:1352: expect (ObjNum 626 snap 146 seq_num 626) 2023-03-29T14:08:00.550 INFO:tasks.rados.rados.0.smithi017.stdout:1351: expect (ObjNum 613 snap 144 seq_num 613) 2023-03-29T14:08:00.670 INFO:tasks.rados.rados.0.smithi017.stdout:1354: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:08:00.692 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~171s 2023-03-29T14:08:00.947 INFO:tasks.rados.rados.0.smithi017.stdout:1353: expect (ObjNum 557 snap 131 seq_num 557) 2023-03-29T14:08:01.229 INFO:tasks.rados.rados.0.smithi017.stdout:1351: done (4 left) 2023-03-29T14:08:01.229 INFO:tasks.rados.rados.0.smithi017.stdout:1352: done (3 left) 2023-03-29T14:08:01.229 INFO:tasks.rados.rados.0.smithi017.stdout:1353: done (2 left) 2023-03-29T14:08:01.229 INFO:tasks.rados.rados.0.smithi017.stdout:1354: done (1 left) 2023-03-29T14:08:01.229 INFO:tasks.rados.rados.0.smithi017.stdout:1355: done (0 left) 2023-03-29T14:08:01.229 INFO:tasks.rados.rados.0.smithi017.stdout:1356: write oid 30 current snap is 155 2023-03-29T14:08:01.229 INFO:tasks.rados.rados.0.smithi017.stdout:1356: seq_num 656 ranges {625392=569702,1637717=745548,2848542=176850} 2023-03-29T14:08:01.235 INFO:tasks.rados.rados.0.smithi017.stdout:1356: writing smithi017107884-30 from 625392 to 1195094 tid 1 2023-03-29T14:08:01.242 INFO:tasks.rados.rados.0.smithi017.stdout:1356: writing smithi017107884-30 from 1637717 to 2383265 tid 2 2023-03-29T14:08:01.245 INFO:tasks.rados.rados.0.smithi017.stdout:1356: writing smithi017107884-30 from 2848542 to 3025392 tid 3 2023-03-29T14:08:01.246 INFO:tasks.rados.rados.0.smithi017.stdout:1357: snap_create 2023-03-29T14:08:01.275 INFO:tasks.rados.rados.0.smithi017.stdout:1356: finishing write tid 1 to smithi017107884-30 2023-03-29T14:08:01.288 INFO:tasks.rados.rados.0.smithi017.stdout:1356: finishing write tid 2 to smithi017107884-30 2023-03-29T14:08:01.290 INFO:tasks.rados.rados.0.smithi017.stdout:1356: finishing write tid 3 to smithi017107884-30 2023-03-29T14:08:01.292 INFO:tasks.rados.rados.0.smithi017.stdout:1356: finishing write tid 4 to smithi017107884-30 2023-03-29T14:08:01.293 INFO:tasks.rados.rados.0.smithi017.stdout:1356: finishing write tid 5 to smithi017107884-30 2023-03-29T14:08:01.294 INFO:tasks.rados.rados.0.smithi017.stdout:1356: oid 30 updating version 0 to 86 2023-03-29T14:08:01.294 INFO:tasks.rados.rados.0.smithi017.stdout:1356: oid 30 version 86 is already newer than 86 2023-03-29T14:08:01.294 INFO:tasks.rados.rados.0.smithi017.stdout:1356: oid 30 version 86 is already newer than 85 2023-03-29T14:08:01.294 INFO:tasks.rados.rados.0.smithi017.stdout:1356: oid 30 version 86 is already newer than 86 2023-03-29T14:08:01.294 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 86 (ObjNum 656 snap 155 seq_num 656) dirty exists 2023-03-29T14:08:01.294 INFO:tasks.rados.rados.0.smithi017.stdout:1356: left oid 30 (ObjNum 656 snap 155 seq_num 656) 2023-03-29T14:08:01.386 INFO:tasks.rados.rados.0.smithi017.stdout:1356: done (1 left) 2023-03-29T14:08:01.386 INFO:tasks.rados.rados.0.smithi017.stdout:1357: done (0 left) 2023-03-29T14:08:01.386 INFO:tasks.rados.rados.0.smithi017.stdout:1358: delete oid 16 current snap is 156 2023-03-29T14:08:01.411 INFO:tasks.rados.rados.0.smithi017.stdout:1358: done (0 left) 2023-03-29T14:08:01.411 INFO:tasks.rados.rados.0.smithi017.stdout:1359: snap_create 2023-03-29T14:08:02.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:02.381 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:02.380+0000 7f53eae6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:02.387+0000 7f53eae6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:02.390 INFO:tasks.rados.rados.0.smithi017.stdout:1359: done (0 left) 2023-03-29T14:08:02.390 INFO:tasks.rados.rados.0.smithi017.stdout:1360: write (excl) oid 50 current snap is 157 2023-03-29T14:08:02.390 INFO:tasks.rados.rados.0.smithi017.stdout:1360: seq_num 658 ranges {659006=720200,2154681=749217} 2023-03-29T14:08:02.398 INFO:tasks.rados.rados.0.smithi017.stdout:1360: writing smithi017107884-50 from 659006 to 1379206 tid 1 2023-03-29T14:08:02.407 INFO:tasks.rados.rados.0.smithi017.stdout:1360: writing smithi017107884-50 from 2154681 to 2903898 tid 2 2023-03-29T14:08:02.407 INFO:tasks.rados.rados.0.smithi017.stdout:1361: read oid 37 snap 155 2023-03-29T14:08:02.407 INFO:tasks.rados.rados.0.smithi017.stdout:1361: expect deleted 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1362: read oid 36 snap 152 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1362: expect (ObjNum 642 snap 152 seq_num 642) 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1363: read oid 2 snap 155 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1363: expect (ObjNum 648 snap 154 seq_num 648) 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1364: read oid 15 snap -1 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1364: expect (ObjNum 653 snap 155 seq_num 653) 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1365: read oid 9 snap -1 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1365: expect (ObjNum 573 snap 133 seq_num 573) 2023-03-29T14:08:02.408 INFO:tasks.rados.rados.0.smithi017.stdout:1366: snap_create 2023-03-29T14:08:02.437 INFO:tasks.rados.rados.0.smithi017.stdout:1365: expect (ObjNum 573 snap 133 seq_num 573) 2023-03-29T14:08:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:02.554 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:02.555 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:02.691 INFO:tasks.rados.rados.0.smithi017.stdout:1363: expect (ObjNum 648 snap 154 seq_num 648) 2023-03-29T14:08:02.913 INFO:tasks.rados.rados.0.smithi017.stdout:1360: finishing write tid 1 to smithi017107884-50 2023-03-29T14:08:02.913 INFO:tasks.rados.rados.0.smithi017.stdout:1364: expect (ObjNum 653 snap 155 seq_num 653) 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1360: finishing write tid 2 to smithi017107884-50 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1360: finishing write tid 3 to smithi017107884-50 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1360: finishing write tid 4 to smithi017107884-50 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1360: oid 50 updating version 0 to 119 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1360: oid 50 version 119 is already newer than 119 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1360: oid 50 version 119 is already newer than 117 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 119 (ObjNum 658 snap 157 seq_num 658) dirty exists 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1360: left oid 50 (ObjNum 658 snap 157 seq_num 658) 2023-03-29T14:08:03.030 INFO:tasks.rados.rados.0.smithi017.stdout:1362: expect (ObjNum 642 snap 152 seq_num 642) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1360: done (6 left) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1361: done (5 left) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1362: done (4 left) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1363: done (3 left) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1364: done (2 left) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1365: done (1 left) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1366: done (0 left) 2023-03-29T14:08:03.393 INFO:tasks.rados.rados.0.smithi017.stdout:1367: snap_remove snap 157 2023-03-29T14:08:04.081 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:04.082 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:04.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:04.209+0000 7f10c0903700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:04.216+0000 7f10c0903700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:04.395 INFO:tasks.rados.rados.0.smithi017.stdout:1367: done (0 left) 2023-03-29T14:08:04.395 INFO:tasks.rados.rados.0.smithi017.stdout:1368: read oid 40 snap -1 2023-03-29T14:08:04.395 INFO:tasks.rados.rados.0.smithi017.stdout:1368: expect deleted 2023-03-29T14:08:04.395 INFO:tasks.rados.rados.0.smithi017.stdout:1369: snap_create 2023-03-29T14:08:04.419 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:04.432 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2023-03-29T14:08:04.393132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175711884,"num_objects":740,"num_object_clones":644,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":3044,"num_read_kb":1740304,"num_write":4176,"num_write_kb":1129542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2203,"ondisk_log_size":2203,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1431500,"kb_used_data":1354392,"kb_used_omap":0,"kb_used_meta":76416,"kb_avail":376055860,"statfs":{"total":386547056640,"available":385081200640,"internally_reserved":0,"allocated":1386897408,"data_stored":1690909104,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":78249984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33548,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":28422,"num_write":46,"num_write_kb":12418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.147277"},"pg_stats":[{"pgid":"3.f","version":"323'94","reported_seq":274,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:01.403966+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:01.403966+0000","last_peered":"2023-03-29T14:08:01.403966+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:01.403966+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067558,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":99,"num_read_kb":60675,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":340,"reported_epoch":334,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:59.299629+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:07:59.299629+0000","last_peered":"2023-03-29T14:07:59.299629+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:07:59.299629+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"325'213","reported_seq":751,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-03-29T14:08:00.266340+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:00.266340+0000","last_peered":"2023-03-29T14:08:00.266340+0000","last_clean":"2023-03-29T14:08:00.266340+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:00.266340+0000","last_undegraded":"2023-03-29T14:08:00.266340+0000","last_fullsized":"2023-03-29T14:08:00.266340+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27396664,"num_objects":74,"num_object_clones":64,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":210130,"num_write":410,"num_write_kb":120584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"331'189","reported_seq":698,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-03-29T14:08:00.277149+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:00.277149+0000","last_peered":"2023-03-29T14:08:00.277149+0000","last_clean":"2023-03-29T14:08:00.277149+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:00.277149+0000","last_undegraded":"2023-03-29T14:08:00.277149+0000","last_fullsized":"2023-03-29T14:08:00.277149+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13177150,"num_objects":78,"num_object_clones":70,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":277,"num_read_kb":168206,"num_write":366,"num_write_kb":102228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"331'188","reported_seq":670,"reported_epoch":334,"state":"active+clean","last_fresh":"2023-03-29T14:07:59.331360+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:07:59.331360+0000","last_peered":"2023-03-29T14:07:59.331360+0000","last_clean":"2023-03-29T14:07:59.331360+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:07:59.331360+0000","last_undegraded":"2023-03-29T14:07:59.331360+0000","last_fullsized":"2023-03-29T14:07:59.331360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12740168,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":239,"num_read_kb":136348,"num_write":372,"num_write_kb":108196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":279,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-03-29T14:08:00.271869+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:00.271869+0000","last_peered":"2023-03-29T14:08:00.271869+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:00.271869+0000","last_undegraded":"2023-03-29T14:08:00.271869+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":523,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:00.281399+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:00.281399+0000","last_peered":"2023-03-29T14:08:00.281399+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:00.281399+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":453,"reported_epoch":335,"state":"active+clean","last_fresh":"2023-03-29T14:08:00.266909+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:00.266909+0000","last_peered":"2023-03-29T14:08:00.266909+0000","last_clean":"2023-03-29T14:08:00.266909+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:00.266909+0000","last_undegraded":"2023-03-29T14:08:00.266909+0000","last_fullsized":"2023-03-29T14:08:00.266909+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":263,"reported_epoch":334,"state":"active+undersized","last_fresh":"2023-03-29T14:07:59.316425+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:07:59.316425+0000","last_peered":"2023-03-29T14:07:59.316425+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:07:59.316425+0000","last_undegraded":"2023-03-29T14:07:59.316425+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"334'85","reported_seq":225,"reported_epoch":334,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:01.282505+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:01.282505+0000","last_peered":"2023-03-29T14:08:01.282505+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:01.282505+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050784,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":36,"num_read_kb":12592,"num_write":96,"num_write_kb":25750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":282,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-03-29T14:08:00.269385+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:00.269385+0000","last_peered":"2023-03-29T14:08:00.269385+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:00.269385+0000","last_undegraded":"2023-03-29T14:08:00.269385+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"334'88","reported_seq":473,"reported_epoch":334,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:00.385333+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:00.385333+0000","last_peered":"2023-03-29T14:08:00.385333+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:00.385333+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":100905,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":124,"reported_epoch":334,"state":"active+undersized","last_fresh":"2023-03-29T14:07:59.312287+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:07:59.312287+0000","last_peered":"2023-03-29T14:07:59.312287+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:07:59.312287+0000","last_undegraded":"2023-03-29T14:07:59.312287+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"325'116","reported_seq":512,"reported_epoch":334,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:59.319340+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:07:59.319340+0000","last_peered":"2023-03-29T14:07:59.319340+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:07:59.319340+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10387254,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":71578,"num_write":224,"num_write_kb":65496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":282,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-03-29T14:08:00.270582+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:00.270582+0000","last_peered":"2023-03-29T14:08:00.270582+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:00.270582+0000","last_undegraded":"2023-03-29T14:08:00.270582+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":488,"reported_epoch":334,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:07:59.302522+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:07:59.302522+0000","last_peered":"2023-03-29T14:07:59.302522+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:07:59.302522+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"316'147","reported_seq":662,"reported_epoch":335,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:00.372553+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:00.372553+0000","last_peered":"2023-03-29T14:08:00.372553+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:00.372553+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244708,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":306,"num_read_kb":211482,"num_write":292,"num_write_kb":83434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":282,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-03-29T14:08:00.279173+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:00.279173+0000","last_peered":"2023-03-29T14:08:00.279173+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:00.279173+0000","last_undegraded":"2023-03-29T14:08:00.279173+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"331'283","reported_seq":910,"reported_epoch":334,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:00.414574+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:00.414574+0000","last_peered":"2023-03-29T14:08:00.414574+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:00.414574+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21413174,"num_objects":100,"num_object_clones":86,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":433,"num_read_kb":258339,"num_write":530,"num_write_kb":152688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":100}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":263,"reported_epoch":334,"state":"active+undersized","last_fresh":"2023-03-29T14:07:59.327533+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:07:59.327533+0000","last_peered":"2023-03-29T14:07:59.327533+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:07:59.327533+0000","last_undegraded":"2023-03-29T14:07:59.327533+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":459,"reported_epoch":334,"state":"active+clean","last_fresh":"2023-03-29T14:07:59.291623+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:07:59.291623+0000","last_peered":"2023-03-29T14:07:59.291623+0000","last_clean":"2023-03-29T14:07:59.291623+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:07:59.291623+0000","last_undegraded":"2023-03-29T14:07:59.291623+0000","last_fullsized":"2023-03-29T14:07:59.291623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"322'191","reported_seq":668,"reported_epoch":334,"state":"active+clean","last_fresh":"2023-03-29T14:08:00.456634+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:00.456634+0000","last_peered":"2023-03-29T14:08:00.456634+0000","last_clean":"2023-03-29T14:08:00.456634+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:00.456634+0000","last_undegraded":"2023-03-29T14:08:00.456634+0000","last_fullsized":"2023-03-29T14:08:00.456634+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":240,"num_read_kb":142866,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":263,"reported_epoch":334,"state":"active+undersized","last_fresh":"2023-03-29T14:07:59.320670+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:07:59.320670+0000","last_peered":"2023-03-29T14:07:59.320670+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:07:59.320670+0000","last_undegraded":"2023-03-29T14:07:59.320670+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":282,"reported_epoch":335,"state":"active+undersized","last_fresh":"2023-03-29T14:08:00.280679+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:00.280679+0000","last_peered":"2023-03-29T14:08:00.280679+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:00.280679+0000","last_undegraded":"2023-03-29T14:08:00.280679+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"329'191","reported_seq":686,"reported_epoch":334,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:00.410483+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:00.410483+0000","last_peered":"2023-03-29T14:08:00.410483+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:00.410483+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20841340,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":252,"num_read_kb":145612,"num_write":368,"num_write_kb":109140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":174531148,"num_objects":736,"num_object_clones":644,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":2902,"num_read_kb":1740184,"num_write":3902,"num_write_kb":1125754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2106,"ondisk_log_size":2106,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":406572,"kb_used_data":385040,"kb_used_omap":0,"kb_used_meta":20864,"kb_avail":93965268,"statfs":{"total":96636764160,"available":96220434432,"internally_reserved":0,"allocated":394280960,"data_stored":479730770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21364736},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":457848,"kb_used_data":435696,"kb_used_omap":0,"kb_used_meta":22144,"kb_avail":93913992,"statfs":{"total":96636764160,"available":96167927808,"internally_reserved":0,"allocated":446152704,"data_stored":549863038,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22675456},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":411136,"kb_used_data":389816,"kb_used_omap":0,"kb_used_meta":21312,"kb_avail":93960704,"statfs":{"total":96636764160,"available":96215760896,"internally_reserved":0,"allocated":399171584,"data_stored":488083071,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21823488},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:04.555 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:04.683+0000 7fa97b88d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:04.687+0000 7fa97b88d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:04.750 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:04.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:04.753 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:05.398 INFO:tasks.rados.rados.0.smithi017.stdout:1368: done (1 left) 2023-03-29T14:08:05.398 INFO:tasks.rados.rados.0.smithi017.stdout:1369: done (0 left) 2023-03-29T14:08:05.398 INFO:tasks.rados.rados.0.smithi017.stdout:1370: snap_remove snap 155 2023-03-29T14:08:06.196 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~176s 2023-03-29T14:08:06.404 INFO:tasks.rados.rados.0.smithi017.stdout:1370: done (0 left) 2023-03-29T14:08:06.404 INFO:tasks.rados.rados.0.smithi017.stdout:1371: snap_create 2023-03-29T14:08:06.754 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:06.881+0000 7f089dec3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:06.886 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:06.886+0000 7f089dec3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:06.948 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:06.950 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:06.951 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:07.405 INFO:tasks.rados.rados.0.smithi017.stdout:1371: done (0 left) 2023-03-29T14:08:07.405 INFO:tasks.rados.rados.0.smithi017.stdout:1372: write (excl) oid 17 current snap is 160 2023-03-29T14:08:07.405 INFO:tasks.rados.rados.0.smithi017.stdout:1372: seq_num 659 ranges {675813=795449,2096810=609228} 2023-03-29T14:08:07.414 INFO:tasks.rados.rados.0.smithi017.stdout:1372: writing smithi017107884-17 from 675813 to 1471262 tid 1 2023-03-29T14:08:07.421 INFO:tasks.rados.rados.0.smithi017.stdout:1372: writing smithi017107884-17 from 2096810 to 2706038 tid 2 2023-03-29T14:08:07.421 INFO:tasks.rados.rados.0.smithi017.stdout:1373: snap_create 2023-03-29T14:08:07.541 INFO:tasks.rados.rados.0.smithi017.stdout:1372: finishing write tid 1 to smithi017107884-17 2023-03-29T14:08:07.591 INFO:tasks.rados.rados.0.smithi017.stdout:1372: finishing write tid 2 to smithi017107884-17 2023-03-29T14:08:07.592 INFO:tasks.rados.rados.0.smithi017.stdout:1372: finishing write tid 3 to smithi017107884-17 2023-03-29T14:08:07.592 INFO:tasks.rados.rados.0.smithi017.stdout:1372: finishing write tid 4 to smithi017107884-17 2023-03-29T14:08:07.593 INFO:tasks.rados.rados.0.smithi017.stdout:1372: oid 17 updating version 0 to 214 2023-03-29T14:08:07.593 INFO:tasks.rados.rados.0.smithi017.stdout:1372: oid 17 updating version 214 to 217 2023-03-29T14:08:07.593 INFO:tasks.rados.rados.0.smithi017.stdout:1372: oid 17 version 217 is already newer than 217 2023-03-29T14:08:07.593 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 217 (ObjNum 659 snap 160 seq_num 659) dirty exists 2023-03-29T14:08:07.593 INFO:tasks.rados.rados.0.smithi017.stdout:1372: left oid 17 (ObjNum 659 snap 160 seq_num 659) 2023-03-29T14:08:08.244 INFO:tasks.rados.rados.0.smithi017.stdout:1372: done (1 left) 2023-03-29T14:08:08.244 INFO:tasks.rados.rados.0.smithi017.stdout:1373: done (0 left) 2023-03-29T14:08:08.244 INFO:tasks.rados.rados.0.smithi017.stdout:1374: delete oid 13 current snap is 161 2023-03-29T14:08:08.414 INFO:tasks.rados.rados.0.smithi017.stdout:1374: done (0 left) 2023-03-29T14:08:08.414 INFO:tasks.rados.rados.0.smithi017.stdout:1375: snap_create 2023-03-29T14:08:08.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:09.080 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:09.079+0000 7f65fa7c1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:09.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:09.083+0000 7f65fa7c1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:09.157 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:09.160 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:09.161 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:09.411 INFO:tasks.rados.rados.0.smithi017.stdout:1375: done (0 left) 2023-03-29T14:08:09.412 INFO:tasks.rados.rados.0.smithi017.stdout:1376: snap_remove snap 152 2023-03-29T14:08:09.439 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:09.439 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:09.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:09.573 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:09.570+0000 7f76ab2dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:09.577 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:09.577+0000 7f76ab2dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:09.781 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:09.783 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:09.795 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2023-03-29T14:08:09.410335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":181829896,"num_objects":740,"num_object_clones":644,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":3076,"num_read_kb":1768369,"num_write":4184,"num_write_kb":1132414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2208,"ondisk_log_size":2208,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1434688,"kb_used_data":1358176,"kb_used_omap":0,"kb_used_meta":76480,"kb_avail":376052672,"statfs":{"total":386547056640,"available":385077936128,"internally_reserved":0,"allocated":1390772224,"data_stored":1694137704,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":78315520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6118012,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":32,"num_read_kb":28065,"num_write":8,"num_write_kb":2872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013639"},"pg_stats":[{"pgid":"3.f","version":"323'94","reported_seq":278,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:05.421011+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:05.421011+0000","last_peered":"2023-03-29T14:08:05.421011+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:05.421011+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067558,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":99,"num_read_kb":60675,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":344,"reported_epoch":338,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:03.411703+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:03.411703+0000","last_peered":"2023-03-29T14:08:03.411703+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:03.411703+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521314,"num_objects":16,"num_object_clones":14,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"325'213","reported_seq":755,"reported_epoch":340,"state":"active+clean","last_fresh":"2023-03-29T14:08:05.418196+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:05.418196+0000","last_peered":"2023-03-29T14:08:05.418196+0000","last_clean":"2023-03-29T14:08:05.418196+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:05.418196+0000","last_undegraded":"2023-03-29T14:08:05.418196+0000","last_fullsized":"2023-03-29T14:08:05.418196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27396664,"num_objects":74,"num_object_clones":64,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":210130,"num_write":410,"num_write_kb":120584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"331'189","reported_seq":705,"reported_epoch":340,"state":"active+clean","last_fresh":"2023-03-29T14:08:05.414602+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:05.414602+0000","last_peered":"2023-03-29T14:08:05.414602+0000","last_clean":"2023-03-29T14:08:05.414602+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:05.414602+0000","last_undegraded":"2023-03-29T14:08:05.414602+0000","last_fullsized":"2023-03-29T14:08:05.414602+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13177150,"num_objects":78,"num_object_clones":70,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":284,"num_read_kb":177245,"num_write":366,"num_write_kb":102228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"331'188","reported_seq":674,"reported_epoch":338,"state":"active+clean","last_fresh":"2023-03-29T14:08:03.425543+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:03.425543+0000","last_peered":"2023-03-29T14:08:03.425543+0000","last_clean":"2023-03-29T14:08:03.425543+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:03.425543+0000","last_undegraded":"2023-03-29T14:08:03.425543+0000","last_fullsized":"2023-03-29T14:08:03.425543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12740168,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":239,"num_read_kb":136348,"num_write":372,"num_write_kb":108196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":283,"reported_epoch":340,"state":"active+undersized","last_fresh":"2023-03-29T14:08:05.416582+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:05.416582+0000","last_peered":"2023-03-29T14:08:05.416582+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:05.416582+0000","last_undegraded":"2023-03-29T14:08:05.416582+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":527,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:05.420241+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:05.420241+0000","last_peered":"2023-03-29T14:08:05.420241+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:05.420241+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":457,"reported_epoch":340,"state":"active+clean","last_fresh":"2023-03-29T14:08:05.408169+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:05.408169+0000","last_peered":"2023-03-29T14:08:05.408169+0000","last_clean":"2023-03-29T14:08:05.408169+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:05.408169+0000","last_undegraded":"2023-03-29T14:08:05.408169+0000","last_fullsized":"2023-03-29T14:08:05.408169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":267,"reported_epoch":338,"state":"active+undersized","last_fresh":"2023-03-29T14:08:03.430490+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:03.430490+0000","last_peered":"2023-03-29T14:08:03.430490+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:03.430490+0000","last_undegraded":"2023-03-29T14:08:03.430490+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"334'86","reported_seq":229,"reported_epoch":338,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:02.457491+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:02.457491+0000","last_peered":"2023-03-29T14:08:02.457491+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:02.457491+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050784,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":12593,"num_write":96,"num_write_kb":25750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":286,"reported_epoch":340,"state":"active+undersized","last_fresh":"2023-03-29T14:08:05.412693+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:05.412693+0000","last_peered":"2023-03-29T14:08:05.412693+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:05.412693+0000","last_undegraded":"2023-03-29T14:08:05.412693+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"334'88","reported_seq":476,"reported_epoch":338,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:02.473735+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:02.473735+0000","last_peered":"2023-03-29T14:08:02.473735+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:02.473735+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":100905,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":338,"state":"active+undersized","last_fresh":"2023-03-29T14:08:03.446451+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:03.446451+0000","last_peered":"2023-03-29T14:08:03.446451+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:03.446451+0000","last_undegraded":"2023-03-29T14:08:03.446451+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"336'119","reported_seq":526,"reported_epoch":338,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:03.448450+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:03.448450+0000","last_peered":"2023-03-29T14:08:03.448450+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:03.448450+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16505266,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":156,"num_read_kb":76102,"num_write":232,"num_write_kb":68368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":286,"reported_epoch":340,"state":"active+undersized","last_fresh":"2023-03-29T14:08:05.419286+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:05.419286+0000","last_peered":"2023-03-29T14:08:05.419286+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:05.419286+0000","last_undegraded":"2023-03-29T14:08:05.419286+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":492,"reported_epoch":338,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:03.433399+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:03.433399+0000","last_peered":"2023-03-29T14:08:03.433399+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:03.433399+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"316'147","reported_seq":666,"reported_epoch":340,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:05.404998+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:05.404998+0000","last_peered":"2023-03-29T14:08:05.404998+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:05.404998+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244708,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":306,"num_read_kb":211482,"num_write":292,"num_write_kb":83434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":286,"reported_epoch":340,"state":"active+undersized","last_fresh":"2023-03-29T14:08:05.402343+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:05.402343+0000","last_peered":"2023-03-29T14:08:05.402343+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:05.402343+0000","last_undegraded":"2023-03-29T14:08:05.402343+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"334'284","reported_seq":919,"reported_epoch":338,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:02.574247+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:02.574247+0000","last_peered":"2023-03-29T14:08:02.574247+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:02.574247+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21413174,"num_objects":100,"num_object_clones":86,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":447,"num_read_kb":272840,"num_write":530,"num_write_kb":152688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":100}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":267,"reported_epoch":338,"state":"active+undersized","last_fresh":"2023-03-29T14:08:03.442568+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:03.442568+0000","last_peered":"2023-03-29T14:08:03.442568+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:03.442568+0000","last_undegraded":"2023-03-29T14:08:03.442568+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":463,"reported_epoch":338,"state":"active+clean","last_fresh":"2023-03-29T14:08:03.423362+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:03.423362+0000","last_peered":"2023-03-29T14:08:03.423362+0000","last_clean":"2023-03-29T14:08:03.423362+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:03.423362+0000","last_undegraded":"2023-03-29T14:08:03.423362+0000","last_fullsized":"2023-03-29T14:08:03.423362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"322'191","reported_seq":672,"reported_epoch":338,"state":"active+clean","last_fresh":"2023-03-29T14:08:03.413627+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:03.413627+0000","last_peered":"2023-03-29T14:08:03.413627+0000","last_clean":"2023-03-29T14:08:03.413627+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:03.413627+0000","last_undegraded":"2023-03-29T14:08:03.413627+0000","last_fullsized":"2023-03-29T14:08:03.413627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":240,"num_read_kb":142866,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":267,"reported_epoch":338,"state":"active+undersized","last_fresh":"2023-03-29T14:08:03.440560+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:03.440560+0000","last_peered":"2023-03-29T14:08:03.440560+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:03.440560+0000","last_undegraded":"2023-03-29T14:08:03.440560+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":286,"reported_epoch":340,"state":"active+undersized","last_fresh":"2023-03-29T14:08:05.417350+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:05.417350+0000","last_peered":"2023-03-29T14:08:05.417350+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:05.417350+0000","last_undegraded":"2023-03-29T14:08:05.417350+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"329'191","reported_seq":689,"reported_epoch":338,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:02.464985+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:02.464985+0000","last_peered":"2023-03-29T14:08:02.464985+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:02.464985+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20841340,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":252,"num_read_kb":145612,"num_write":368,"num_write_kb":109140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":180649160,"num_objects":736,"num_object_clones":644,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":2934,"num_read_kb":1768249,"num_write":3910,"num_write_kb":1128626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2111,"ondisk_log_size":2111,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":406016,"kb_used_data":385080,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":93965824,"statfs":{"total":96636764160,"available":96221003776,"internally_reserved":0,"allocated":394321920,"data_stored":479752175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":460120,"kb_used_data":437968,"kb_used_omap":0,"kb_used_meta":22144,"kb_avail":93911720,"statfs":{"total":96636764160,"available":96165601280,"internally_reserved":0,"allocated":448479232,"data_stored":551583696,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22675456},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":412608,"kb_used_data":391288,"kb_used_omap":0,"kb_used_meta":21312,"kb_avail":93959232,"statfs":{"total":96636764160,"available":96214253568,"internally_reserved":0,"allocated":400678912,"data_stored":489569608,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21823488},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:09.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.9 3.5 3.3 3.0 1.0 3.1 3.6 2023-03-29T14:08:09.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:09.925+0000 7f9070d6c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:09.940+0000 7f9070d6c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:10.145 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3,3.5,3.6] on osd.0 to force-backfill; instructing pg(s) [1.0,3.0] on osd.1 to force-backfill; instructing pg(s) [3.9] on osd.3 to force-backfill; 2023-03-29T14:08:10.145 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:08:10.415 INFO:tasks.rados.rados.0.smithi017.stdout:1376: done (0 left) 2023-03-29T14:08:10.416 INFO:tasks.rados.rados.0.smithi017.stdout:1377: delete oid 1 current snap is 162 2023-03-29T14:08:10.509 INFO:tasks.rados.rados.0.smithi017.stdout:1377: done (0 left) 2023-03-29T14:08:10.509 INFO:tasks.rados.rados.0.smithi017.stdout:1378: delete oid 38 current snap is 162 2023-03-29T14:08:10.627 INFO:tasks.rados.rados.0.smithi017.stdout:1378: done (0 left) 2023-03-29T14:08:10.627 INFO:tasks.rados.rados.0.smithi017.stdout:1379: write (excl) oid 33 current snap is 162 2023-03-29T14:08:10.627 INFO:tasks.rados.rados.0.smithi017.stdout:1379: seq_num 663 ranges {743041=412798,1930738=616566,3040503=102538} 2023-03-29T14:08:10.633 INFO:tasks.rados.rados.0.smithi017.stdout:1379: writing smithi017107884-33 from 743041 to 1155839 tid 1 2023-03-29T14:08:10.641 INFO:tasks.rados.rados.0.smithi017.stdout:1379: writing smithi017107884-33 from 1930738 to 2547304 tid 2 2023-03-29T14:08:10.644 INFO:tasks.rados.rados.0.smithi017.stdout:1379: writing smithi017107884-33 from 3040503 to 3143041 tid 3 2023-03-29T14:08:10.644 INFO:tasks.rados.rados.0.smithi017.stdout:1380: snap_create 2023-03-29T14:08:10.681 INFO:tasks.rados.rados.0.smithi017.stdout:1379: finishing write tid 1 to smithi017107884-33 2023-03-29T14:08:10.713 INFO:tasks.rados.rados.0.smithi017.stdout:1379: finishing write tid 2 to smithi017107884-33 2023-03-29T14:08:10.714 INFO:tasks.rados.rados.0.smithi017.stdout:1379: finishing write tid 3 to smithi017107884-33 2023-03-29T14:08:10.716 INFO:tasks.rados.rados.0.smithi017.stdout:1379: finishing write tid 4 to smithi017107884-33 2023-03-29T14:08:10.718 INFO:tasks.rados.rados.0.smithi017.stdout:1379: finishing write tid 5 to smithi017107884-33 2023-03-29T14:08:10.718 INFO:tasks.rados.rados.0.smithi017.stdout:1379: oid 33 updating version 0 to 194 2023-03-29T14:08:10.718 INFO:tasks.rados.rados.0.smithi017.stdout:1379: oid 33 updating version 194 to 197 2023-03-29T14:08:10.718 INFO:tasks.rados.rados.0.smithi017.stdout:1379: oid 33 version 197 is already newer than 197 2023-03-29T14:08:10.718 INFO:tasks.rados.rados.0.smithi017.stdout:1379: oid 33 version 197 is already newer than 197 2023-03-29T14:08:10.718 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 197 (ObjNum 663 snap 162 seq_num 663) dirty exists 2023-03-29T14:08:10.718 INFO:tasks.rados.rados.0.smithi017.stdout:1379: left oid 33 (ObjNum 663 snap 162 seq_num 663) 2023-03-29T14:08:11.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:11.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:11.290+0000 7faa8d104700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:11.294 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:11.294+0000 7faa8d104700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:11.358 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:11.360 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:11.361 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:11.422 INFO:tasks.rados.rados.0.smithi017.stdout:1379: done (1 left) 2023-03-29T14:08:11.422 INFO:tasks.rados.rados.0.smithi017.stdout:1380: done (0 left) 2023-03-29T14:08:11.422 INFO:tasks.rados.rados.0.smithi017.stdout:1381: read oid 17 snap -1 2023-03-29T14:08:11.422 INFO:tasks.rados.rados.0.smithi017.stdout:1381: expect (ObjNum 659 snap 160 seq_num 659) 2023-03-29T14:08:11.422 INFO:tasks.rados.rados.0.smithi017.stdout:1382: delete oid 23 current snap is 163 2023-03-29T14:08:11.460 INFO:tasks.rados.rados.0.smithi017.stdout:1381: expect (ObjNum 659 snap 160 seq_num 659) 2023-03-29T14:08:11.612 INFO:tasks.rados.rados.0.smithi017.stdout:1381: done (1 left) 2023-03-29T14:08:11.612 INFO:tasks.rados.rados.0.smithi017.stdout:1382: done (0 left) 2023-03-29T14:08:11.612 INFO:tasks.rados.rados.0.smithi017.stdout:1383: write oid 44 current snap is 163 2023-03-29T14:08:11.612 INFO:tasks.rados.rados.0.smithi017.stdout:1383: seq_num 665 ranges {776655=563296,2047702=620235,3095349=81306} 2023-03-29T14:08:11.619 INFO:tasks.rados.rados.0.smithi017.stdout:1383: writing smithi017107884-44 from 776655 to 1339951 tid 1 2023-03-29T14:08:11.626 INFO:tasks.rados.rados.0.smithi017.stdout:1383: writing smithi017107884-44 from 2047702 to 2667937 tid 2 2023-03-29T14:08:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:1383: writing smithi017107884-44 from 3095349 to 3176655 tid 3 2023-03-29T14:08:11.629 INFO:tasks.rados.rados.0.smithi017.stdout:1384: delete oid 21 current snap is 163 2023-03-29T14:08:11.631 INFO:tasks.rados.rados.0.smithi017.stdout:1384: done (1 left) 2023-03-29T14:08:11.631 INFO:tasks.rados.rados.0.smithi017.stdout:1385: write oid 32 current snap is 163 2023-03-29T14:08:11.631 INFO:tasks.rados.rados.0.smithi017.stdout:1385: seq_num 667 ranges {410269=713794,1648313=507551,2917489=292780} 2023-03-29T14:08:11.637 INFO:tasks.rados.rados.0.smithi017.stdout:1385: writing smithi017107884-32 from 410269 to 1124063 tid 1 2023-03-29T14:08:11.643 INFO:tasks.rados.rados.0.smithi017.stdout:1385: writing smithi017107884-32 from 1648313 to 2155864 tid 2 2023-03-29T14:08:11.649 INFO:tasks.rados.rados.0.smithi017.stdout:1385: writing smithi017107884-32 from 2917489 to 3210269 tid 3 2023-03-29T14:08:11.649 INFO:tasks.rados.rados.0.smithi017.stdout:1386: snap_remove snap 158 2023-03-29T14:08:11.699 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~182s 2023-03-29T14:08:12.426 INFO:tasks.rados.rados.0.smithi017.stdout:1386: done (2 left) 2023-03-29T14:08:12.426 INFO:tasks.rados.rados.0.smithi017.stdout:1387: read oid 34 snap -1 2023-03-29T14:08:12.426 INFO:tasks.rados.rados.0.smithi017.stdout:1387: expect deleted 2023-03-29T14:08:12.426 INFO:tasks.rados.rados.0.smithi017.stdout:1388: read oid 9 snap -1 2023-03-29T14:08:12.426 INFO:tasks.rados.rados.0.smithi017.stdout:1388: expect (ObjNum 573 snap 133 seq_num 573) 2023-03-29T14:08:12.426 INFO:tasks.rados.rados.0.smithi017.stdout:1389: snap_remove snap 156 2023-03-29T14:08:13.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:13.428 INFO:tasks.rados.rados.0.smithi017.stdout:1389: done (4 left) 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1390: snap_create 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: finishing write tid 1 to smithi017107884-32 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: finishing write tid 2 to smithi017107884-32 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: finishing write tid 3 to smithi017107884-32 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: finishing write tid 4 to smithi017107884-32 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: finishing write tid 5 to smithi017107884-32 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: oid 32 updating version 0 to 288 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: oid 32 updating version 288 to 290 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: oid 32 version 290 is already newer than 290 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: oid 32 version 290 is already newer than 289 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 290 (ObjNum 667 snap 163 seq_num 667) dirty exists 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1385: left oid 32 (ObjNum 667 snap 163 seq_num 667) 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1383: finishing write tid 1 to smithi017107884-44 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1383: finishing write tid 2 to smithi017107884-44 2023-03-29T14:08:13.429 INFO:tasks.rados.rados.0.smithi017.stdout:1383: finishing write tid 3 to smithi017107884-44 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1383: finishing write tid 4 to smithi017107884-44 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1383: finishing write tid 5 to smithi017107884-44 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1383: oid 44 updating version 0 to 94 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1383: oid 44 version 94 is already newer than 94 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1383: oid 44 version 94 is already newer than 94 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1383: oid 44 version 94 is already newer than 94 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 94 (ObjNum 665 snap 163 seq_num 665) dirty exists 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1383: left oid 44 (ObjNum 665 snap 163 seq_num 665) 2023-03-29T14:08:13.430 INFO:tasks.rados.rados.0.smithi017.stdout:1388: expect (ObjNum 573 snap 133 seq_num 573) 2023-03-29T14:08:13.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:13.503+0000 7ff3a235c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:13.508+0000 7ff3a235c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:13.576 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:13.578 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:13.579 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:14.435 INFO:tasks.rados.rados.0.smithi017.stdout:1383: done (4 left) 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1385: done (3 left) 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1387: done (2 left) 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1388: done (1 left) 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1390: done (0 left) 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1391: read oid 43 snap -1 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1391: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1392: read oid 5 snap -1 2023-03-29T14:08:14.436 INFO:tasks.rados.rados.0.smithi017.stdout:1392: expect (ObjNum 555 snap 129 seq_num 555) 2023-03-29T14:08:14.437 INFO:tasks.rados.rados.0.smithi017.stdout:1393: read oid 48 snap -1 2023-03-29T14:08:14.437 INFO:tasks.rados.rados.0.smithi017.stdout:1393: expect (ObjNum 605 snap 143 seq_num 605) 2023-03-29T14:08:14.437 INFO:tasks.rados.rados.0.smithi017.stdout:1394: delete oid 16 current snap is 164 2023-03-29T14:08:14.476 INFO:tasks.rados.rados.0.smithi017.stdout:1394: done (3 left) 2023-03-29T14:08:14.476 INFO:tasks.rados.rados.0.smithi017.stdout:1395: snap_remove snap 162 2023-03-29T14:08:15.158 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:15.158 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:15.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:15.285+0000 7f8a02127700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:15.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:15.292+0000 7f8a02127700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:15.438 INFO:tasks.rados.rados.0.smithi017.stdout:1395: done (3 left) 2023-03-29T14:08:15.439 INFO:tasks.rados.rados.0.smithi017.stdout:1396: read oid 8 snap -1 2023-03-29T14:08:15.439 INFO:tasks.rados.rados.0.smithi017.stdout:1396: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:08:15.439 INFO:tasks.rados.rados.0.smithi017.stdout:1397: read oid 23 snap -1 2023-03-29T14:08:15.439 INFO:tasks.rados.rados.0.smithi017.stdout:1397: expect deleted 2023-03-29T14:08:15.439 INFO:tasks.rados.rados.0.smithi017.stdout:1398: write (excl) oid 9 current snap is 164 2023-03-29T14:08:15.439 INFO:tasks.rados.rados.0.smithi017.stdout:1398: seq_num 669 ranges {443883=464292,1365277=511220,2572335=671548} 2023-03-29T14:08:15.443 INFO:tasks.rados.rados.0.smithi017.stdout:1398: writing smithi017107884-9 from 443883 to 908175 tid 1 2023-03-29T14:08:15.449 INFO:tasks.rados.rados.0.smithi017.stdout:1398: writing smithi017107884-9 from 1365277 to 1876497 tid 2 2023-03-29T14:08:15.456 INFO:tasks.rados.rados.0.smithi017.stdout:1398: writing smithi017107884-9 from 2572335 to 3243883 tid 3 2023-03-29T14:08:15.456 INFO:tasks.rados.rados.0.smithi017.stdout:1399: write (excl) oid 6 current snap is 164 2023-03-29T14:08:15.456 INFO:tasks.rados.rados.0.smithi017.stdout:1399: seq_num 670 ranges {460690=655894,1540112=771231,2716111=539778} 2023-03-29T14:08:15.462 INFO:tasks.rados.rados.0.smithi017.stdout:1399: writing smithi017107884-6 from 460690 to 1116584 tid 1 2023-03-29T14:08:15.469 INFO:tasks.rados.rados.0.smithi017.stdout:1399: writing smithi017107884-6 from 1540112 to 2311343 tid 2 2023-03-29T14:08:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1399: writing smithi017107884-6 from 2716111 to 3255889 tid 3 2023-03-29T14:08:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1400: read oid 22 snap 159 2023-03-29T14:08:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1400: expect (ObjNum 597 snap 141 seq_num 597) 2023-03-29T14:08:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1401: read oid 21 snap 163 2023-03-29T14:08:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1401: expect deleted 2023-03-29T14:08:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1402: write oid 36 current snap is 164 2023-03-29T14:08:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1402: seq_num 671 ranges {477497=731143,1998594=514889,3143534=133963} 2023-03-29T14:08:15.481 INFO:tasks.rados.rados.0.smithi017.stdout:1402: writing smithi017107884-36 from 477497 to 1208640 tid 1 2023-03-29T14:08:15.488 INFO:tasks.rados.rados.0.smithi017.stdout:1402: writing smithi017107884-36 from 1998594 to 2513483 tid 2 2023-03-29T14:08:15.493 INFO:tasks.rados.rados.0.smithi017.stdout:1402: writing smithi017107884-36 from 3143534 to 3277497 tid 3 2023-03-29T14:08:15.493 INFO:tasks.rados.rados.0.smithi017.stdout:1403: read oid 18 snap -1 2023-03-29T14:08:15.493 INFO:tasks.rados.rados.0.smithi017.stdout:1403: expect deleted 2023-03-29T14:08:15.493 INFO:tasks.rados.rados.0.smithi017.stdout:1404: write (excl) oid 20 current snap is 164 2023-03-29T14:08:15.493 INFO:tasks.rados.rados.0.smithi017.stdout:1393: expect (ObjNum 605 snap 143 seq_num 605) 2023-03-29T14:08:15.502 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:15.504 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:15.517 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2023-03-29T14:08:15.437560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":174972508,"num_objects":746,"num_object_clones":650,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":3091,"num_read_kb":1776020,"num_write":4202,"num_write_kb":1137374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2218,"ondisk_log_size":2218,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1439104,"kb_used_data":1362464,"kb_used_omap":0,"kb_used_meta":76608,"kb_avail":376048256,"statfs":{"total":386547056640,"available":385073414144,"internally_reserved":0,"allocated":1395163136,"data_stored":1699249397,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":78446592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1445336,"num_objects":2,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7650,"num_write":8,"num_write_kb":2744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016683"},"pg_stats":[{"pgid":"3.f","version":"323'94","reported_seq":283,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:10.540620+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:10.540620+0000","last_peered":"2023-03-29T14:08:10.540620+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:10.540620+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067558,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":99,"num_read_kb":60675,"num_write":106,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"276'37","reported_seq":349,"reported_epoch":344,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:10.589357+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:10.589357+0000","last_peered":"2023-03-29T14:08:10.589357+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:10.589357+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"341'217","reported_seq":770,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-03-29T14:08:11.455744+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:11.455744+0000","last_peered":"2023-03-29T14:08:11.455744+0000","last_clean":"2023-03-29T14:08:11.455744+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:11.455744+0000","last_undegraded":"2023-03-29T14:08:11.455744+0000","last_fullsized":"2023-03-29T14:08:11.455744+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25951328,"num_objects":76,"num_object_clones":66,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":314,"num_read_kb":217780,"num_write":418,"num_write_kb":123328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"331'189","reported_seq":710,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-03-29T14:08:10.543052+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:10.543052+0000","last_peered":"2023-03-29T14:08:10.543052+0000","last_clean":"2023-03-29T14:08:10.543052+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:10.543052+0000","last_undegraded":"2023-03-29T14:08:10.543052+0000","last_fullsized":"2023-03-29T14:08:10.543052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13177150,"num_objects":78,"num_object_clones":70,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":284,"num_read_kb":177245,"num_write":366,"num_write_kb":102228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"331'188","reported_seq":677,"reported_epoch":344,"state":"active+clean","last_fresh":"2023-03-29T14:08:09.468313+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:09.468313+0000","last_peered":"2023-03-29T14:08:09.468313+0000","last_clean":"2023-03-29T14:08:09.468313+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:09.468313+0000","last_undegraded":"2023-03-29T14:08:09.468313+0000","last_fullsized":"2023-03-29T14:08:09.468313+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12740168,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":239,"num_read_kb":136348,"num_write":372,"num_write_kb":108196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":288,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-03-29T14:08:10.531112+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:10.531112+0000","last_peered":"2023-03-29T14:08:10.531112+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:10.531112+0000","last_undegraded":"2023-03-29T14:08:10.531112+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":532,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:10.546414+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:10.546414+0000","last_peered":"2023-03-29T14:08:10.546414+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:10.546414+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":462,"reported_epoch":345,"state":"active+clean","last_fresh":"2023-03-29T14:08:10.544901+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:10.544901+0000","last_peered":"2023-03-29T14:08:10.544901+0000","last_clean":"2023-03-29T14:08:10.544901+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:10.544901+0000","last_undegraded":"2023-03-29T14:08:10.544901+0000","last_fullsized":"2023-03-29T14:08:10.544901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":270,"reported_epoch":344,"state":"active+undersized","last_fresh":"2023-03-29T14:08:09.487338+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:09.487338+0000","last_peered":"2023-03-29T14:08:09.487338+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:09.487338+0000","last_undegraded":"2023-03-29T14:08:09.487338+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"334'86","reported_seq":232,"reported_epoch":344,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:09.533346+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:09.533346+0000","last_peered":"2023-03-29T14:08:09.533346+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:09.533346+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050784,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":12593,"num_write":96,"num_write_kb":25750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":291,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-03-29T14:08:10.529862+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:10.529862+0000","last_peered":"2023-03-29T14:08:10.529862+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:10.529862+0000","last_undegraded":"2023-03-29T14:08:10.529862+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"334'88","reported_seq":479,"reported_epoch":344,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:09.503579+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:09.503579+0000","last_peered":"2023-03-29T14:08:09.503579+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:09.503579+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":100905,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":131,"reported_epoch":344,"state":"active+undersized","last_fresh":"2023-03-29T14:08:09.454639+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:09.454639+0000","last_peered":"2023-03-29T14:08:09.454639+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:09.454639+0000","last_undegraded":"2023-03-29T14:08:09.454639+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"336'119","reported_seq":529,"reported_epoch":344,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:09.459350+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:09.459350+0000","last_peered":"2023-03-29T14:08:09.459350+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:09.459350+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16505266,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":156,"num_read_kb":76102,"num_write":232,"num_write_kb":68368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":291,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-03-29T14:08:10.537212+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:10.537212+0000","last_peered":"2023-03-29T14:08:10.537212+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:10.537212+0000","last_undegraded":"2023-03-29T14:08:10.537212+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":495,"reported_epoch":344,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:09.466349+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:09.466349+0000","last_peered":"2023-03-29T14:08:09.466349+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:09.466349+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"316'147","reported_seq":671,"reported_epoch":345,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:10.538006+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:10.538006+0000","last_peered":"2023-03-29T14:08:10.538006+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:10.538006+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244708,"num_objects":52,"num_object_clones":44,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":306,"num_read_kb":211482,"num_write":292,"num_write_kb":83434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-03-29T14:08:10.545496+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:10.545496+0000","last_peered":"2023-03-29T14:08:10.545496+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:10.545496+0000","last_undegraded":"2023-03-29T14:08:10.545496+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"334'284","reported_seq":922,"reported_epoch":344,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:09.529585+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:09.529585+0000","last_peered":"2023-03-29T14:08:09.529585+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:09.529585+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21413174,"num_objects":100,"num_object_clones":86,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":447,"num_read_kb":272840,"num_write":530,"num_write_kb":152688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":100}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":270,"reported_epoch":344,"state":"active+undersized","last_fresh":"2023-03-29T14:08:09.435642+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:09.435642+0000","last_peered":"2023-03-29T14:08:09.435642+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:09.435642+0000","last_undegraded":"2023-03-29T14:08:09.435642+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"301'90","reported_seq":466,"reported_epoch":344,"state":"active+clean","last_fresh":"2023-03-29T14:08:09.471461+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:09.471461+0000","last_peered":"2023-03-29T14:08:09.471461+0000","last_clean":"2023-03-29T14:08:09.471461+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:09.471461+0000","last_undegraded":"2023-03-29T14:08:09.471461+0000","last_fullsized":"2023-03-29T14:08:09.471461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4000330,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":124,"num_read_kb":70179,"num_write":184,"num_write_kb":52354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"322'191","reported_seq":676,"reported_epoch":344,"state":"active+clean","last_fresh":"2023-03-29T14:08:09.426403+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:09.426403+0000","last_peered":"2023-03-29T14:08:09.426403+0000","last_clean":"2023-03-29T14:08:09.426403+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:09.426403+0000","last_undegraded":"2023-03-29T14:08:09.426403+0000","last_fullsized":"2023-03-29T14:08:09.426403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":240,"num_read_kb":142866,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":270,"reported_epoch":344,"state":"active+undersized","last_fresh":"2023-03-29T14:08:09.445344+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:09.445344+0000","last_peered":"2023-03-29T14:08:09.445344+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:09.445344+0000","last_undegraded":"2023-03-29T14:08:09.445344+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":291,"reported_epoch":345,"state":"active+undersized","last_fresh":"2023-03-29T14:08:10.539336+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:10.539336+0000","last_peered":"2023-03-29T14:08:10.539336+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:10.539336+0000","last_undegraded":"2023-03-29T14:08:10.539336+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"344'197","reported_seq":703,"reported_epoch":344,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:10.702082+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:10.702082+0000","last_peered":"2023-03-29T14:08:10.702082+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:10.702082+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950602,"num_objects":70,"num_object_clones":62,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":145613,"num_write":378,"num_write_kb":111356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":173791772,"num_objects":742,"num_object_clones":650,"num_object_copies":2226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":40,"num_read":2949,"num_read_kb":1775900,"num_write":3928,"num_write_kb":1133586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2121,"ondisk_log_size":2121,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":407440,"kb_used_data":386504,"kb_used_omap":0,"kb_used_meta":20928,"kb_avail":93964400,"statfs":{"total":96636764160,"available":96219545600,"internally_reserved":0,"allocated":395780096,"data_stored":481453216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21430272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":461616,"kb_used_data":439400,"kb_used_omap":0,"kb_used_meta":22208,"kb_avail":93910224,"statfs":{"total":96636764160,"available":96164069376,"internally_reserved":0,"allocated":449945600,"data_stored":553289022,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22740992},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":414104,"kb_used_data":392720,"kb_used_omap":0,"kb_used_meta":21376,"kb_avail":93957736,"statfs":{"total":96636764160,"available":96212721664,"internally_reserved":0,"allocated":402145280,"data_stored":491274934,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21889024},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:15.580 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:15.636 INFO:tasks.rados.rados.0.smithi017.stdout:1391: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:15.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:15.719+0000 7f24048a2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:15.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:15.724+0000 7f24048a2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:15.769 INFO:tasks.rados.rados.0.smithi017.stdout:1392: expect (ObjNum 555 snap 129 seq_num 555) 2023-03-29T14:08:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:15.793 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:15.794 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:16.005 INFO:tasks.rados.rados.0.smithi017.stdout:1400: expect (ObjNum 597 snap 141 seq_num 597) 2023-03-29T14:08:16.137 INFO:tasks.rados.rados.0.smithi017.stdout:1398: finishing write tid 1 to smithi017107884-9 2023-03-29T14:08:16.137 INFO:tasks.rados.rados.0.smithi017.stdout:1396: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:08:16.414 INFO:tasks.rados.rados.0.smithi017.stdout:1399: finishing write tid 1 to smithi017107884-6 2023-03-29T14:08:16.415 INFO:tasks.rados.rados.0.smithi017.stdout:1398: finishing write tid 2 to smithi017107884-9 2023-03-29T14:08:16.415 INFO:tasks.rados.rados.0.smithi017.stdout:1402: finishing write tid 1 to smithi017107884-36 2023-03-29T14:08:16.415 INFO:tasks.rados.rados.0.smithi017.stdout:1398: finishing write tid 3 to smithi017107884-9 2023-03-29T14:08:16.415 INFO:tasks.rados.rados.0.smithi017.stdout:1398: finishing write tid 4 to smithi017107884-9 2023-03-29T14:08:16.415 INFO:tasks.rados.rados.0.smithi017.stdout:1404: seq_num 672 ranges {494304=406392,1540723=774900,2770957=523347} 2023-03-29T14:08:16.419 INFO:tasks.rados.rados.0.smithi017.stdout:1404: writing smithi017107884-20 from 494304 to 900696 tid 1 2023-03-29T14:08:16.427 INFO:tasks.rados.rados.0.smithi017.stdout:1404: writing smithi017107884-20 from 1540723 to 2315623 tid 2 2023-03-29T14:08:16.434 INFO:tasks.rados.rados.0.smithi017.stdout:1404: writing smithi017107884-20 from 2770957 to 3294304 tid 3 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1391: done (11 left) 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1392: done (10 left) 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1393: done (9 left) 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1396: done (8 left) 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1397: done (7 left) 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1400: done (6 left) 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1403: done (5 left) 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1405: write oid 39 current snap is 164 2023-03-29T14:08:16.435 INFO:tasks.rados.rados.0.smithi017.stdout:1405: seq_num 673 ranges {511111=481641,1599205=518558,2682027=489947} 2023-03-29T14:08:16.440 INFO:tasks.rados.rados.0.smithi017.stdout:1405: writing smithi017107884-39 from 511111 to 992752 tid 1 2023-03-29T14:08:16.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: writing smithi017107884-39 from 1599205 to 2117763 tid 2 2023-03-29T14:08:16.455 INFO:tasks.rados.rados.0.smithi017.stdout:1405: writing smithi017107884-39 from 2682027 to 3171974 tid 3 2023-03-29T14:08:16.455 INFO:tasks.rados.rados.0.smithi017.stdout:1406: write (excl) oid 11 current snap is 164 2023-03-29T14:08:16.455 INFO:tasks.rados.rados.0.smithi017.stdout:1402: finishing write tid 2 to smithi017107884-36 2023-03-29T14:08:16.455 INFO:tasks.rados.rados.0.smithi017.stdout:1406: seq_num 674 ranges {527918=556890,1657687=778569,3225803=102115} 2023-03-29T14:08:16.461 INFO:tasks.rados.rados.0.smithi017.stdout:1406: writing smithi017107884-11 from 527918 to 1084808 tid 1 2023-03-29T14:08:16.471 INFO:tasks.rados.rados.0.smithi017.stdout:1406: writing smithi017107884-11 from 1657687 to 2436256 tid 2 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:1406: writing smithi017107884-11 from 3225803 to 3327918 tid 3 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:1407: write oid 35 current snap is 164 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:1398: finishing write tid 5 to smithi017107884-9 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:1398: oid 9 updating version 0 to 194 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:1398: oid 9 version 194 is already newer than 194 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:1398: oid 9 version 194 is already newer than 190 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:1398: oid 9 version 194 is already newer than 194 2023-03-29T14:08:16.476 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 194 (ObjNum 669 snap 164 seq_num 669) dirty exists 2023-03-29T14:08:16.477 INFO:tasks.rados.rados.0.smithi017.stdout:1398: left oid 9 (ObjNum 669 snap 164 seq_num 669) 2023-03-29T14:08:16.477 INFO:tasks.rados.rados.0.smithi017.stdout:1402: finishing write tid 3 to smithi017107884-36 2023-03-29T14:08:16.477 INFO:tasks.rados.rados.0.smithi017.stdout:1407: seq_num 675 ranges {544725=632139,1716169=522227,2736873=512491} 2023-03-29T14:08:16.484 INFO:tasks.rados.rados.0.smithi017.stdout:1407: writing smithi017107884-35 from 544725 to 1176864 tid 1 2023-03-29T14:08:16.490 INFO:tasks.rados.rados.0.smithi017.stdout:1407: writing smithi017107884-35 from 1716169 to 2238396 tid 2 2023-03-29T14:08:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1407: writing smithi017107884-35 from 2736873 to 3249364 tid 3 2023-03-29T14:08:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1398: done (7 left) 2023-03-29T14:08:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1408: read oid 17 snap -1 2023-03-29T14:08:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1408: expect (ObjNum 659 snap 160 seq_num 659) 2023-03-29T14:08:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1409: write (excl) oid 41 current snap is 164 2023-03-29T14:08:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1402: finishing write tid 4 to smithi017107884-36 2023-03-29T14:08:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1409: seq_num 676 ranges {561532=707388,2058298=782238} 2023-03-29T14:08:16.504 INFO:tasks.rados.rados.0.smithi017.stdout:1409: writing smithi017107884-41 from 561532 to 1268920 tid 1 2023-03-29T14:08:16.515 INFO:tasks.rados.rados.0.smithi017.stdout:1409: writing smithi017107884-41 from 2058298 to 2840536 tid 2 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1410: write oid 22 current snap is 164 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1402: finishing write tid 5 to smithi017107884-36 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1402: oid 36 updating version 0 to 124 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1402: oid 36 version 124 is already newer than 124 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1402: oid 36 version 124 is already newer than 124 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1402: oid 36 version 124 is already newer than 124 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 124 (ObjNum 671 snap 164 seq_num 671) dirty exists 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1402: left oid 36 (ObjNum 671 snap 164 seq_num 671) 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1399: finishing write tid 2 to smithi017107884-6 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1399: finishing write tid 3 to smithi017107884-6 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1399: finishing write tid 4 to smithi017107884-6 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1399: finishing write tid 5 to smithi017107884-6 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1399: oid 6 updating version 0 to 154 2023-03-29T14:08:16.516 INFO:tasks.rados.rados.0.smithi017.stdout:1399: oid 6 version 154 is already newer than 154 2023-03-29T14:08:16.517 INFO:tasks.rados.rados.0.smithi017.stdout:1399: oid 6 version 154 is already newer than 154 2023-03-29T14:08:16.517 INFO:tasks.rados.rados.0.smithi017.stdout:1399: oid 6 version 154 is already newer than 150 2023-03-29T14:08:16.517 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 154 (ObjNum 670 snap 164 seq_num 670) dirty exists 2023-03-29T14:08:16.517 INFO:tasks.rados.rados.0.smithi017.stdout:1399: left oid 6 (ObjNum 670 snap 164 seq_num 670) 2023-03-29T14:08:16.517 INFO:tasks.rados.rados.0.smithi017.stdout:1410: seq_num 677 ranges {578339=498990,1833133=525896,2791719=586620} 2023-03-29T14:08:16.521 INFO:tasks.rados.rados.0.smithi017.stdout:1410: writing smithi017107884-22 from 578339 to 1077329 tid 1 2023-03-29T14:08:16.529 INFO:tasks.rados.rados.0.smithi017.stdout:1410: writing smithi017107884-22 from 1833133 to 2359029 tid 2 2023-03-29T14:08:16.535 INFO:tasks.rados.rados.0.smithi017.stdout:1410: writing smithi017107884-22 from 2791719 to 3378339 tid 3 2023-03-29T14:08:16.535 INFO:tasks.rados.rados.0.smithi017.stdout:1399: done (9 left) 2023-03-29T14:08:16.535 INFO:tasks.rados.rados.0.smithi017.stdout:1402: done (8 left) 2023-03-29T14:08:16.535 INFO:tasks.rados.rados.0.smithi017.stdout:1411: snap_remove snap 160 2023-03-29T14:08:17.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~187s 2023-03-29T14:08:17.447 INFO:tasks.rados.rados.0.smithi017.stdout:1411: done (8 left) 2023-03-29T14:08:17.447 INFO:tasks.rados.rados.0.smithi017.stdout:1412: snap_create 2023-03-29T14:08:17.447 INFO:tasks.rados.rados.0.smithi017.stdout:1404: finishing write tid 1 to smithi017107884-20 2023-03-29T14:08:17.447 INFO:tasks.rados.rados.0.smithi017.stdout:1405: finishing write tid 1 to smithi017107884-39 2023-03-29T14:08:17.447 INFO:tasks.rados.rados.0.smithi017.stdout:1405: finishing write tid 2 to smithi017107884-39 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: finishing write tid 3 to smithi017107884-39 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1406: finishing write tid 1 to smithi017107884-11 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: finishing write tid 4 to smithi017107884-39 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: finishing write tid 5 to smithi017107884-39 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: oid 39 updating version 0 to 222 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: oid 39 version 222 is already newer than 222 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: oid 39 version 222 is already newer than 222 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: oid 39 version 222 is already newer than 222 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 222 (ObjNum 673 snap 164 seq_num 673) dirty exists 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1405: left oid 39 (ObjNum 673 snap 164 seq_num 673) 2023-03-29T14:08:17.448 INFO:tasks.rados.rados.0.smithi017.stdout:1408: expect (ObjNum 659 snap 160 seq_num 659) 2023-03-29T14:08:17.598 INFO:tasks.rados.rados.0.smithi017.stdout:1404: finishing write tid 2 to smithi017107884-20 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: finishing write tid 3 to smithi017107884-20 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: finishing write tid 4 to smithi017107884-20 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: finishing write tid 5 to smithi017107884-20 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: oid 20 updating version 0 to 193 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: oid 20 version 193 is already newer than 193 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: oid 20 version 193 is already newer than 193 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: oid 20 version 193 is already newer than 189 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 193 (ObjNum 672 snap 164 seq_num 672) dirty exists 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1404: left oid 20 (ObjNum 672 snap 164 seq_num 672) 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1407: finishing write tid 1 to smithi017107884-35 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1407: finishing write tid 2 to smithi017107884-35 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1410: finishing write tid 1 to smithi017107884-22 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1406: finishing write tid 2 to smithi017107884-11 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1406: finishing write tid 3 to smithi017107884-11 2023-03-29T14:08:17.599 INFO:tasks.rados.rados.0.smithi017.stdout:1406: finishing write tid 4 to smithi017107884-11 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1406: finishing write tid 5 to smithi017107884-11 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1406: oid 11 updating version 0 to 294 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1406: oid 11 version 294 is already newer than 294 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1406: oid 11 version 294 is already newer than 291 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1406: oid 11 version 294 is already newer than 294 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 11 v 294 (ObjNum 674 snap 164 seq_num 674) dirty exists 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1406: left oid 11 (ObjNum 674 snap 164 seq_num 674) 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: finishing write tid 3 to smithi017107884-35 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: finishing write tid 4 to smithi017107884-35 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: finishing write tid 5 to smithi017107884-35 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: oid 35 updating version 0 to 99 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: oid 35 version 99 is already newer than 99 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: oid 35 version 99 is already newer than 99 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: oid 35 version 99 is already newer than 99 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 99 (ObjNum 675 snap 164 seq_num 675) dirty exists 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1407: left oid 35 (ObjNum 675 snap 164 seq_num 675) 2023-03-29T14:08:17.600 INFO:tasks.rados.rados.0.smithi017.stdout:1409: finishing write tid 1 to smithi017107884-41 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: finishing write tid 2 to smithi017107884-22 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: finishing write tid 3 to smithi017107884-22 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: finishing write tid 4 to smithi017107884-22 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: finishing write tid 5 to smithi017107884-22 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: oid 22 updating version 0 to 101 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: oid 22 version 101 is already newer than 101 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: oid 22 version 101 is already newer than 101 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: oid 22 version 101 is already newer than 101 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 101 (ObjNum 677 snap 164 seq_num 677) dirty exists 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1410: left oid 22 (ObjNum 677 snap 164 seq_num 677) 2023-03-29T14:08:17.601 INFO:tasks.rados.rados.0.smithi017.stdout:1409: finishing write tid 2 to smithi017107884-41 2023-03-29T14:08:17.602 INFO:tasks.rados.rados.0.smithi017.stdout:1409: finishing write tid 3 to smithi017107884-41 2023-03-29T14:08:17.602 INFO:tasks.rados.rados.0.smithi017.stdout:1409: finishing write tid 4 to smithi017107884-41 2023-03-29T14:08:17.602 INFO:tasks.rados.rados.0.smithi017.stdout:1409: oid 41 updating version 0 to 195 2023-03-29T14:08:17.602 INFO:tasks.rados.rados.0.smithi017.stdout:1409: oid 41 version 195 is already newer than 193 2023-03-29T14:08:17.602 INFO:tasks.rados.rados.0.smithi017.stdout:1409: oid 41 version 195 is already newer than 195 2023-03-29T14:08:17.602 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 195 (ObjNum 676 snap 164 seq_num 676) dirty exists 2023-03-29T14:08:17.602 INFO:tasks.rados.rados.0.smithi017.stdout:1409: left oid 41 (ObjNum 676 snap 164 seq_num 676) 2023-03-29T14:08:17.795 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:17.923+0000 7f1d9c265700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:17.928+0000 7f1d9c265700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:18.004 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:18.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1401: done (8 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1404: done (7 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1405: done (6 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1406: done (5 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1407: done (4 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1408: done (3 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1409: done (2 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1410: done (1 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1412: done (0 left) 2023-03-29T14:08:18.451 INFO:tasks.rados.rados.0.smithi017.stdout:1413: snap_create 2023-03-29T14:08:19.452 INFO:tasks.rados.rados.0.smithi017.stdout:1413: done (0 left) 2023-03-29T14:08:19.452 INFO:tasks.rados.rados.0.smithi017.stdout:1414: delete oid 25 current snap is 166 2023-03-29T14:08:19.577 INFO:tasks.rados.rados.0.smithi017.stdout:1414: done (0 left) 2023-03-29T14:08:19.577 INFO:tasks.rados.rados.0.smithi017.stdout:1415: write oid 35 current snap is 166 2023-03-29T14:08:19.577 INFO:tasks.rados.rados.0.smithi017.stdout:1415: seq_num 679 ranges {611953=649488,1950097=529565,3246565=165388} 2023-03-29T14:08:19.584 INFO:tasks.rados.rados.0.smithi017.stdout:1415: writing smithi017107884-35 from 611953 to 1261441 tid 1 2023-03-29T14:08:19.591 INFO:tasks.rados.rados.0.smithi017.stdout:1415: writing smithi017107884-35 from 1950097 to 2479662 tid 2 2023-03-29T14:08:19.596 INFO:tasks.rados.rados.0.smithi017.stdout:1415: writing smithi017107884-35 from 3246565 to 3411953 tid 3 2023-03-29T14:08:19.596 INFO:tasks.rados.rados.0.smithi017.stdout:1416: read oid 12 snap -1 2023-03-29T14:08:19.596 INFO:tasks.rados.rados.0.smithi017.stdout:1416: expect (ObjNum 557 snap 131 seq_num 557) 2023-03-29T14:08:19.596 INFO:tasks.rados.rados.0.smithi017.stdout:1417: write (excl) oid 10 current snap is 166 2023-03-29T14:08:19.596 INFO:tasks.rados.rados.0.smithi017.stdout:1417: seq_num 680 ranges {628760=724737,1892226=673223,3157635=271125} 2023-03-29T14:08:19.603 INFO:tasks.rados.rados.0.smithi017.stdout:1417: writing smithi017107884-10 from 628760 to 1353497 tid 1 2023-03-29T14:08:19.610 INFO:tasks.rados.rados.0.smithi017.stdout:1417: writing smithi017107884-10 from 1892226 to 2565449 tid 2 2023-03-29T14:08:19.615 INFO:tasks.rados.rados.0.smithi017.stdout:1417: writing smithi017107884-10 from 3157635 to 3428760 tid 3 2023-03-29T14:08:19.615 INFO:tasks.rados.rados.0.smithi017.stdout:1418: write oid 2 current snap is 166 2023-03-29T14:08:19.615 INFO:tasks.rados.rados.0.smithi017.stdout:1418: seq_num 681 ranges {645567=799986,1950708=533234,3185058=260509} 2023-03-29T14:08:19.624 INFO:tasks.rados.rados.0.smithi017.stdout:1418: writing smithi017107884-2 from 645567 to 1445553 tid 1 2023-03-29T14:08:19.630 INFO:tasks.rados.rados.0.smithi017.stdout:1418: writing smithi017107884-2 from 1950708 to 2483942 tid 2 2023-03-29T14:08:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:1418: writing smithi017107884-2 from 3185058 to 3445567 tid 3 2023-03-29T14:08:19.635 INFO:tasks.rados.rados.0.smithi017.stdout:1419: write oid 40 current snap is 166 2023-03-29T14:08:19.636 INFO:tasks.rados.rados.0.smithi017.stdout:1419: seq_num 682 ranges {662374=475235,1609190=676892,2812481=624101} 2023-03-29T14:08:19.641 INFO:tasks.rados.rados.0.smithi017.stdout:1419: writing smithi017107884-40 from 662374 to 1137609 tid 1 2023-03-29T14:08:19.649 INFO:tasks.rados.rados.0.smithi017.stdout:1419: writing smithi017107884-40 from 1609190 to 2286082 tid 2 2023-03-29T14:08:19.657 INFO:tasks.rados.rados.0.smithi017.stdout:1419: writing smithi017107884-40 from 2812481 to 3436582 tid 3 2023-03-29T14:08:19.657 INFO:tasks.rados.rados.0.smithi017.stdout:1420: read oid 1 snap -1 2023-03-29T14:08:19.657 INFO:tasks.rados.rados.0.smithi017.stdout:1420: expect deleted 2023-03-29T14:08:19.657 INFO:tasks.rados.rados.0.smithi017.stdout:1421: delete oid 26 current snap is 166 2023-03-29T14:08:19.682 INFO:tasks.rados.rados.0.smithi017.stdout:1415: finishing write tid 1 to smithi017107884-35 2023-03-29T14:08:19.685 INFO:tasks.rados.rados.0.smithi017.stdout:1417: finishing write tid 1 to smithi017107884-10 2023-03-29T14:08:19.689 INFO:tasks.rados.rados.0.smithi017.stdout:1415: finishing write tid 2 to smithi017107884-35 2023-03-29T14:08:19.713 INFO:tasks.rados.rados.0.smithi017.stdout:1415: finishing write tid 3 to smithi017107884-35 2023-03-29T14:08:19.726 INFO:tasks.rados.rados.0.smithi017.stdout:1419: finishing write tid 1 to smithi017107884-40 2023-03-29T14:08:19.740 INFO:tasks.rados.rados.0.smithi017.stdout:1416: expect (ObjNum 557 snap 131 seq_num 557) 2023-03-29T14:08:20.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: finishing write tid 2 to smithi017107884-40 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1415: finishing write tid 4 to smithi017107884-35 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1415: finishing write tid 5 to smithi017107884-35 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1415: oid 35 updating version 0 to 104 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1415: oid 35 version 104 is already newer than 103 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1415: oid 35 version 104 is already newer than 104 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1415: oid 35 version 104 is already newer than 104 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 104 (ObjNum 679 snap 166 seq_num 679) dirty exists 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1415: left oid 35 (ObjNum 679 snap 166 seq_num 679) 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: finishing write tid 3 to smithi017107884-40 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: finishing write tid 4 to smithi017107884-40 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: finishing write tid 5 to smithi017107884-40 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: oid 40 updating version 0 to 200 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: oid 40 version 200 is already newer than 200 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: oid 40 version 200 is already newer than 200 2023-03-29T14:08:20.031 INFO:tasks.rados.rados.0.smithi017.stdout:1419: oid 40 version 200 is already newer than 200 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 200 (ObjNum 682 snap 166 seq_num 682) dirty exists 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1419: left oid 40 (ObjNum 682 snap 166 seq_num 682) 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1418: finishing write tid 1 to smithi017107884-2 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1415: done (6 left) 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1416: done (5 left) 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1419: done (4 left) 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1420: done (3 left) 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1421: done (2 left) 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1422: write oid 1 current snap is 166 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1418: finishing write tid 2 to smithi017107884-2 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1418: finishing write tid 3 to smithi017107884-2 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1418: finishing write tid 4 to smithi017107884-2 2023-03-29T14:08:20.032 INFO:tasks.rados.rados.0.smithi017.stdout:1422: seq_num 684 ranges {695988=625733,2009801=680561,3150974=345014} 2023-03-29T14:08:20.037 INFO:tasks.rados.rados.0.smithi017.stdout:1422: writing smithi017107884-1 from 695988 to 1321721 tid 1 2023-03-29T14:08:20.045 INFO:tasks.rados.rados.0.smithi017.stdout:1422: writing smithi017107884-1 from 2009801 to 2690362 tid 2 2023-03-29T14:08:20.050 INFO:tasks.rados.rados.0.smithi017.stdout:1422: writing smithi017107884-1 from 3150974 to 3495988 tid 3 2023-03-29T14:08:20.050 INFO:tasks.rados.rados.0.smithi017.stdout:1423: read oid 36 snap -1 2023-03-29T14:08:20.050 INFO:tasks.rados.rados.0.smithi017.stdout:1423: expect (ObjNum 671 snap 164 seq_num 671) 2023-03-29T14:08:20.050 INFO:tasks.rados.rados.0.smithi017.stdout:1424: snap_remove snap 165 2023-03-29T14:08:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:20.148+0000 7f7c86486700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:20.153+0000 7f7c86486700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:20.215 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:20.218 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:20.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1424: done (4 left) 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1425: write (excl) oid 40 current snap is 166 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1418: finishing write tid 5 to smithi017107884-2 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1418: oid 2 updating version 0 to 299 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1418: oid 2 version 299 is already newer than 299 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1418: oid 2 version 299 is already newer than 299 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1418: oid 2 version 299 is already newer than 299 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 2 v 299 (ObjNum 681 snap 166 seq_num 681) dirty exists 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1418: left oid 2 (ObjNum 681 snap 166 seq_num 681) 2023-03-29T14:08:20.461 INFO:tasks.rados.rados.0.smithi017.stdout:1425: seq_num 685 ranges {712795=417335,1784636=540572,2894750=457917} 2023-03-29T14:08:20.467 INFO:tasks.rados.rados.0.smithi017.stdout:1425: writing smithi017107884-40 from 712795 to 1130130 tid 1 2023-03-29T14:08:20.474 INFO:tasks.rados.rados.0.smithi017.stdout:1425: writing smithi017107884-40 from 1784636 to 2325208 tid 2 2023-03-29T14:08:20.480 INFO:tasks.rados.rados.0.smithi017.stdout:1425: writing smithi017107884-40 from 2894750 to 3352667 tid 3 2023-03-29T14:08:20.480 INFO:tasks.rados.rados.0.smithi017.stdout:1418: done (4 left) 2023-03-29T14:08:20.480 INFO:tasks.rados.rados.0.smithi017.stdout:1426: delete oid 21 current snap is 166 2023-03-29T14:08:20.480 INFO:tasks.rados.rados.0.smithi017.stdout:1417: finishing write tid 2 to smithi017107884-10 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: finishing write tid 3 to smithi017107884-10 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: finishing write tid 4 to smithi017107884-10 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: finishing write tid 5 to smithi017107884-10 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: oid 10 updating version 0 to 227 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: oid 10 version 227 is already newer than 227 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: oid 10 version 227 is already newer than 227 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: oid 10 version 227 is already newer than 223 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 227 (ObjNum 680 snap 166 seq_num 680) dirty exists 2023-03-29T14:08:20.481 INFO:tasks.rados.rados.0.smithi017.stdout:1417: left oid 10 (ObjNum 680 snap 166 seq_num 680) 2023-03-29T14:08:20.482 INFO:tasks.rados.rados.0.smithi017.stdout:1422: finishing write tid 1 to smithi017107884-1 2023-03-29T14:08:20.482 INFO:tasks.rados.rados.0.smithi017.stdout:1422: finishing write tid 2 to smithi017107884-1 2023-03-29T14:08:20.482 INFO:tasks.rados.rados.0.smithi017.stdout:1422: finishing write tid 3 to smithi017107884-1 2023-03-29T14:08:20.482 INFO:tasks.rados.rados.0.smithi017.stdout:1423: expect (ObjNum 671 snap 164 seq_num 671) 2023-03-29T14:08:20.521 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:20.522 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:20.522 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:20.653 INFO:tasks.rados.rados.0.smithi017.stdout:1422: finishing write tid 4 to smithi017107884-1 2023-03-29T14:08:20.653 INFO:tasks.rados.rados.0.smithi017.stdout:1422: finishing write tid 5 to smithi017107884-1 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1422: oid 1 updating version 0 to 201 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1422: oid 1 version 201 is already newer than 201 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1422: oid 1 version 201 is already newer than 200 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1422: oid 1 version 201 is already newer than 201 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 201 (ObjNum 684 snap 166 seq_num 684) dirty exists 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1422: left oid 1 (ObjNum 684 snap 166 seq_num 684) 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1417: done (4 left) 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1422: done (3 left) 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1423: done (2 left) 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1426: done (1 left) 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1427: delete oid 37 current snap is 166 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1425: finishing write tid 1 to smithi017107884-40 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1425: finishing write tid 2 to smithi017107884-40 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1425: finishing write tid 3 to smithi017107884-40 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1425: finishing write tid 4 to smithi017107884-40 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1425: finishing write tid 5 to smithi017107884-40 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1425: oid 40 updating version 0 to 204 2023-03-29T14:08:20.654 INFO:tasks.rados.rados.0.smithi017.stdout:1425: oid 40 version 204 is already newer than 204 2023-03-29T14:08:20.655 INFO:tasks.rados.rados.0.smithi017.stdout:1425: oid 40 version 204 is already newer than 201 2023-03-29T14:08:20.655 INFO:tasks.rados.rados.0.smithi017.stdout:1425: oid 40 version 204 is already newer than 204 2023-03-29T14:08:20.655 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 204 (ObjNum 685 snap 166 seq_num 685) dirty exists 2023-03-29T14:08:20.655 INFO:tasks.rados.rados.0.smithi017.stdout:1425: left oid 40 (ObjNum 685 snap 166 seq_num 685) 2023-03-29T14:08:20.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:20.657+0000 7f160d866700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:20.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:20.664+0000 7f160d866700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:20.667 INFO:tasks.rados.rados.0.smithi017.stdout:1425: done (1 left) 2023-03-29T14:08:20.667 INFO:tasks.rados.rados.0.smithi017.stdout:1427: done (0 left) 2023-03-29T14:08:20.667 INFO:tasks.rados.rados.0.smithi017.stdout:1428: write (excl) oid 42 current snap is 166 2023-03-29T14:08:20.667 INFO:tasks.rados.rados.0.smithi017.stdout:1428: seq_num 688 ranges {763216=643082,1843729=687899,3260666=302550} 2023-03-29T14:08:20.675 INFO:tasks.rados.rados.0.smithi017.stdout:1428: writing smithi017107884-42 from 763216 to 1406298 tid 1 2023-03-29T14:08:20.681 INFO:tasks.rados.rados.0.smithi017.stdout:1428: writing smithi017107884-42 from 1843729 to 2531628 tid 2 2023-03-29T14:08:20.688 INFO:tasks.rados.rados.0.smithi017.stdout:1428: writing smithi017107884-42 from 3260666 to 3563216 tid 3 2023-03-29T14:08:20.688 INFO:tasks.rados.rados.0.smithi017.stdout:1429: snap_remove snap 163 2023-03-29T14:08:20.870 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:20.872 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:20.884 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2023-03-29T14:08:20.458460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184631956,"num_objects":758,"num_object_clones":662,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":3145,"num_read_kb":1818500,"num_write":4266,"num_write_kb":1157490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2255,"ondisk_log_size":2255,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1453544,"kb_used_data":1376192,"kb_used_omap":0,"kb_used_meta":76864,"kb_avail":376033816,"statfs":{"total":386547056640,"available":385058627584,"internally_reserved":0,"allocated":1409220608,"data_stored":1717245337,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":78708736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9659448,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":54,"num_read_kb":42480,"num_write":64,"num_write_kb":20116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.292016"},"pg_stats":[{"pgid":"3.f","version":"349'98","reported_seq":294,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:16.534217+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:16.534217+0000","last_peered":"2023-03-29T14:08:16.534217+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:16.534217+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718058,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":107,"num_read_kb":65939,"num_write":110,"num_write_kb":32516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"344'39","reported_seq":353,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:14.537433+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:14.537433+0000","last_peered":"2023-03-29T14:08:14.537433+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:14.537433+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"349'221","reported_seq":784,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-03-29T14:08:16.466584+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:16.466584+0000","last_peered":"2023-03-29T14:08:16.466584+0000","last_clean":"2023-03-29T14:08:16.466584+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:16.466584+0000","last_undegraded":"2023-03-29T14:08:16.466584+0000","last_fullsized":"2023-03-29T14:08:16.466584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27127818,"num_objects":78,"num_object_clones":68,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":321,"num_read_kb":224134,"num_write":428,"num_write_kb":126242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"349'194","reported_seq":725,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-03-29T14:08:15.581550+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:15.581550+0000","last_peered":"2023-03-29T14:08:15.581550+0000","last_clean":"2023-03-29T14:08:15.581550+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:15.581550+0000","last_undegraded":"2023-03-29T14:08:15.581550+0000","last_fullsized":"2023-03-29T14:08:15.581550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12404094,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":294,"num_read_kb":184686,"num_write":376,"num_write_kb":105448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"331'188","reported_seq":681,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-03-29T14:08:14.490289+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:14.490289+0000","last_peered":"2023-03-29T14:08:14.490289+0000","last_clean":"2023-03-29T14:08:14.490289+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:14.490289+0000","last_undegraded":"2023-03-29T14:08:14.490289+0000","last_fullsized":"2023-03-29T14:08:14.490289+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12740168,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":239,"num_read_kb":136348,"num_write":372,"num_write_kb":108196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":291,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.459651+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:14.459651+0000","last_peered":"2023-03-29T14:08:14.459651+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:14.459651+0000","last_undegraded":"2023-03-29T14:08:14.459651+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":535,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:14.455890+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:14.455890+0000","last_peered":"2023-03-29T14:08:14.455890+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:14.455890+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":40,"num_object_clones":34,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":142,"num_read_kb":67874,"num_write":206,"num_write_kb":58952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":40}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":465,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-03-29T14:08:14.466020+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:14.466020+0000","last_peered":"2023-03-29T14:08:14.466020+0000","last_clean":"2023-03-29T14:08:14.466020+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:14.466020+0000","last_undegraded":"2023-03-29T14:08:14.466020+0000","last_fullsized":"2023-03-29T14:08:14.466020+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":274,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.514502+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:14.514502+0000","last_peered":"2023-03-29T14:08:14.514502+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:14.514502+0000","last_undegraded":"2023-03-29T14:08:14.514502+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"334'86","reported_seq":235,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:13.484493+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:13.484493+0000","last_peered":"2023-03-29T14:08:13.484493+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:13.484493+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050784,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":12593,"num_write":96,"num_write_kb":25750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":294,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.454623+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:14.454623+0000","last_peered":"2023-03-29T14:08:14.454623+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:14.454623+0000","last_undegraded":"2023-03-29T14:08:14.454623+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"334'88","reported_seq":482,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:13.494514+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:13.494514+0000","last_peered":"2023-03-29T14:08:13.494514+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:13.494514+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":100905,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.473267+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:14.473267+0000","last_peered":"2023-03-29T14:08:14.473267+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:14.473267+0000","last_undegraded":"2023-03-29T14:08:14.473267+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"349'124","reported_seq":545,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:15.683121+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:15.683121+0000","last_peered":"2023-03-29T14:08:15.683121+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:15.683121+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17480072,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":82510,"num_write":242,"num_write_kb":71066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":294,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.450723+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:14.450723+0000","last_peered":"2023-03-29T14:08:14.450723+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:14.450723+0000","last_undegraded":"2023-03-29T14:08:14.450723+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":499,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:14.522322+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:14.522322+0000","last_peered":"2023-03-29T14:08:14.522322+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:14.522322+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"349'154","reported_seq":684,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:15.624073+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:15.624073+0000","last_peered":"2023-03-29T14:08:15.624073+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:15.624073+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16236354,"num_objects":54,"num_object_clones":46,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":310,"num_read_kb":211483,"num_write":302,"num_write_kb":87280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":54}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":294,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.460393+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:14.460393+0000","last_peered":"2023-03-29T14:08:14.460393+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:14.460393+0000","last_undegraded":"2023-03-29T14:08:14.460393+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"345'290","reported_seq":935,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:13.477457+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:13.477457+0000","last_peered":"2023-03-29T14:08:13.477457+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:13.477457+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23698926,"num_objects":102,"num_object_clones":88,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":449,"num_read_kb":272841,"num_write":540,"num_write_kb":155648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":102}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":274,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.476238+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:14.476238+0000","last_peered":"2023-03-29T14:08:14.476238+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:14.476238+0000","last_undegraded":"2023-03-29T14:08:14.476238+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"345'94","reported_seq":479,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-03-29T14:08:14.532519+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:14.532519+0000","last_peered":"2023-03-29T14:08:14.532519+0000","last_clean":"2023-03-29T14:08:14.532519+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:14.532519+0000","last_undegraded":"2023-03-29T14:08:14.532519+0000","last_fullsized":"2023-03-29T14:08:14.532519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10353640,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":70180,"num_write":194,"num_write_kb":54828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"322'191","reported_seq":683,"reported_epoch":349,"state":"active+clean","last_fresh":"2023-03-29T14:08:14.597223+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:14.597223+0000","last_peered":"2023-03-29T14:08:14.597223+0000","last_clean":"2023-03-29T14:08:14.597223+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:14.597223+0000","last_undegraded":"2023-03-29T14:08:14.597223+0000","last_fullsized":"2023-03-29T14:08:14.597223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378768,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":247,"num_read_kb":150244,"num_write":352,"num_write_kb":102078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.510566+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:14.510566+0000","last_peered":"2023-03-29T14:08:14.510566+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:14.510566+0000","last_undegraded":"2023-03-29T14:08:14.510566+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":294,"reported_epoch":349,"state":"active+undersized","last_fresh":"2023-03-29T14:08:14.467128+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:14.467128+0000","last_peered":"2023-03-29T14:08:14.467128+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:14.467128+0000","last_undegraded":"2023-03-29T14:08:14.467128+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"344'197","reported_seq":709,"reported_epoch":349,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:15.585476+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:15.585476+0000","last_peered":"2023-03-29T14:08:15.585476+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:15.585476+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950602,"num_objects":70,"num_object_clones":62,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":264,"num_read_kb":155245,"num_write":378,"num_write_kb":111356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":183451220,"num_objects":754,"num_object_clones":662,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":38,"num_read":3003,"num_read_kb":1818380,"num_write":3992,"num_write_kb":1153702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2158,"ondisk_log_size":2158,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":413892,"kb_used_data":392828,"kb_used_omap":0,"kb_used_meta":21056,"kb_avail":93957948,"statfs":{"total":96636764160,"available":96212938752,"internally_reserved":0,"allocated":402255872,"data_stored":488919788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21561344},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":466032,"kb_used_data":443296,"kb_used_omap":0,"kb_used_meta":22272,"kb_avail":93905808,"statfs":{"total":96636764160,"available":96159547392,"internally_reserved":0,"allocated":453935104,"data_stored":557507467,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22806528},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":417676,"kb_used_data":396228,"kb_used_omap":0,"kb_used_meta":21440,"kb_avail":93954164,"statfs":{"total":96636764160,"available":96209063936,"internally_reserved":0,"allocated":405737472,"data_stored":497585857,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":21954560},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:21.466 INFO:tasks.rados.rados.0.smithi017.stdout:1429: done (1 left) 2023-03-29T14:08:21.466 INFO:tasks.rados.rados.0.smithi017.stdout:1430: write oid 48 current snap is 166 2023-03-29T14:08:21.466 INFO:tasks.rados.rados.0.smithi017.stdout:1428: finishing write tid 1 to smithi017107884-42 2023-03-29T14:08:21.466 INFO:tasks.rados.rados.0.smithi017.stdout:1430: seq_num 689 ranges {780023=718331,1902211=547910,3004442=575581} 2023-03-29T14:08:21.474 INFO:tasks.rados.rados.0.smithi017.stdout:1430: writing smithi017107884-48 from 780023 to 1498354 tid 1 2023-03-29T14:08:21.479 INFO:tasks.rados.rados.0.smithi017.stdout:1430: writing smithi017107884-48 from 1902211 to 2450121 tid 2 2023-03-29T14:08:21.486 INFO:tasks.rados.rados.0.smithi017.stdout:1430: writing smithi017107884-48 from 3004442 to 3580023 tid 3 2023-03-29T14:08:21.486 INFO:tasks.rados.rados.0.smithi017.stdout:1431: read oid 20 snap -1 2023-03-29T14:08:21.486 INFO:tasks.rados.rados.0.smithi017.stdout:1431: expect (ObjNum 672 snap 164 seq_num 672) 2023-03-29T14:08:21.486 INFO:tasks.rados.rados.0.smithi017.stdout:1432: read oid 19 snap 161 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1432: expect (ObjNum 591 snap 139 seq_num 591) 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1433: read oid 36 snap -1 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1433: expect (ObjNum 671 snap 164 seq_num 671) 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1434: read oid 21 snap 164 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1434: expect deleted 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1435: read oid 43 snap -1 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1435: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1436: write (excl) oid 31 current snap is 166 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: finishing write tid 2 to smithi017107884-42 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: finishing write tid 3 to smithi017107884-42 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: finishing write tid 4 to smithi017107884-42 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: finishing write tid 5 to smithi017107884-42 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: oid 42 updating version 0 to 205 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: oid 42 updating version 205 to 208 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: oid 42 version 208 is already newer than 208 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:1428: oid 42 version 208 is already newer than 208 2023-03-29T14:08:21.487 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 208 (ObjNum 688 snap 166 seq_num 688) dirty exists 2023-03-29T14:08:21.488 INFO:tasks.rados.rados.0.smithi017.stdout:1428: left oid 42 (ObjNum 688 snap 166 seq_num 688) 2023-03-29T14:08:21.488 INFO:tasks.rados.rados.0.smithi017.stdout:1436: seq_num 690 ranges {796830=793580,2360693=691568} 2023-03-29T14:08:21.495 INFO:tasks.rados.rados.0.smithi017.stdout:1436: writing smithi017107884-31 from 796830 to 1590410 tid 1 2023-03-29T14:08:21.503 INFO:tasks.rados.rados.0.smithi017.stdout:1436: writing smithi017107884-31 from 2360693 to 3052261 tid 2 2023-03-29T14:08:21.503 INFO:tasks.rados.rados.0.smithi017.stdout:1428: done (7 left) 2023-03-29T14:08:21.503 INFO:tasks.rados.rados.0.smithi017.stdout:1437: write (excl) oid 9 current snap is 166 2023-03-29T14:08:21.503 INFO:tasks.rados.rados.0.smithi017.stdout:1437: seq_num 691 ranges {413637=468829,1619175=435226,2542935=758255} 2023-03-29T14:08:21.507 INFO:tasks.rados.rados.0.smithi017.stdout:1437: writing smithi017107884-9 from 413637 to 882466 tid 1 2023-03-29T14:08:21.514 INFO:tasks.rados.rados.0.smithi017.stdout:1437: writing smithi017107884-9 from 1619175 to 2054401 tid 2 2023-03-29T14:08:21.521 INFO:tasks.rados.rados.0.smithi017.stdout:1437: writing smithi017107884-9 from 2542935 to 3301190 tid 3 2023-03-29T14:08:21.521 INFO:tasks.rados.rados.0.smithi017.stdout:1438: write (excl) oid 3 current snap is 166 2023-03-29T14:08:21.521 INFO:tasks.rados.rados.0.smithi017.stdout:1438: seq_num 692 ranges {430444=660431,1677657=695237,2970358=569527} 2023-03-29T14:08:21.527 INFO:tasks.rados.rados.0.smithi017.stdout:1438: writing smithi017107884-3 from 430444 to 1090875 tid 1 2023-03-29T14:08:21.534 INFO:tasks.rados.rados.0.smithi017.stdout:1438: writing smithi017107884-3 from 1677657 to 2372894 tid 2 2023-03-29T14:08:21.541 INFO:tasks.rados.rados.0.smithi017.stdout:1438: writing smithi017107884-3 from 2970358 to 3539885 tid 3 2023-03-29T14:08:21.541 INFO:tasks.rados.rados.0.smithi017.stdout:1439: read oid 16 snap 164 2023-03-29T14:08:21.541 INFO:tasks.rados.rados.0.smithi017.stdout:1439: expect deleted 2023-03-29T14:08:21.541 INFO:tasks.rados.rados.0.smithi017.stdout:1440: snap_create 2023-03-29T14:08:21.617 INFO:tasks.rados.rados.0.smithi017.stdout:1430: finishing write tid 1 to smithi017107884-48 2023-03-29T14:08:21.625 INFO:tasks.rados.rados.0.smithi017.stdout:1430: finishing write tid 2 to smithi017107884-48 2023-03-29T14:08:21.654 INFO:tasks.rados.rados.0.smithi017.stdout:1432: expect (ObjNum 591 snap 139 seq_num 591) 2023-03-29T14:08:21.816 INFO:tasks.rados.rados.0.smithi017.stdout:1431: expect (ObjNum 672 snap 164 seq_num 672) 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1438: finishing write tid 1 to smithi017107884-3 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1430: finishing write tid 3 to smithi017107884-48 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1430: finishing write tid 4 to smithi017107884-48 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1430: finishing write tid 5 to smithi017107884-48 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1430: oid 48 updating version 0 to 232 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1430: oid 48 version 232 is already newer than 232 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1430: oid 48 version 232 is already newer than 232 2023-03-29T14:08:22.049 INFO:tasks.rados.rados.0.smithi017.stdout:1430: oid 48 version 232 is already newer than 232 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 232 (ObjNum 689 snap 166 seq_num 689) dirty exists 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1430: left oid 48 (ObjNum 689 snap 166 seq_num 689) 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1436: finishing write tid 1 to smithi017107884-31 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: finishing write tid 2 to smithi017107884-3 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: finishing write tid 3 to smithi017107884-3 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: finishing write tid 4 to smithi017107884-3 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: finishing write tid 5 to smithi017107884-3 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: oid 3 updating version 0 to 300 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: oid 3 updating version 300 to 304 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: oid 3 version 304 is already newer than 304 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: oid 3 version 304 is already newer than 304 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 304 (ObjNum 692 snap 166 seq_num 692) dirty exists 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1438: left oid 3 (ObjNum 692 snap 166 seq_num 692) 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1437: finishing write tid 1 to smithi017107884-9 2023-03-29T14:08:22.050 INFO:tasks.rados.rados.0.smithi017.stdout:1433: expect (ObjNum 671 snap 164 seq_num 671) 2023-03-29T14:08:22.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:22.221 INFO:tasks.rados.rados.0.smithi017.stdout:1435: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1436: finishing write tid 2 to smithi017107884-31 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1436: finishing write tid 3 to smithi017107884-31 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1436: finishing write tid 4 to smithi017107884-31 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1436: oid 31 updating version 0 to 116 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1436: oid 31 updating version 116 to 119 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1436: oid 31 version 119 is already newer than 119 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 119 (ObjNum 690 snap 166 seq_num 690) dirty exists 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1436: left oid 31 (ObjNum 690 snap 166 seq_num 690) 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1437: finishing write tid 2 to smithi017107884-9 2023-03-29T14:08:22.348 INFO:tasks.rados.rados.0.smithi017.stdout:1437: finishing write tid 3 to smithi017107884-9 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:1437: finishing write tid 4 to smithi017107884-9 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:1437: finishing write tid 5 to smithi017107884-9 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:1437: oid 9 updating version 0 to 213 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:1437: oid 9 version 213 is already newer than 213 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:1437: oid 9 version 213 is already newer than 209 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:1437: oid 9 version 213 is already newer than 213 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 213 (ObjNum 691 snap 166 seq_num 691) dirty exists 2023-03-29T14:08:22.349 INFO:tasks.rados.rados.0.smithi017.stdout:1437: left oid 9 (ObjNum 691 snap 166 seq_num 691) 2023-03-29T14:08:22.349 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:22.348+0000 7f469ae4b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:22.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:22.352+0000 7f469ae4b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:22.417 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:22.418 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1430: done (10 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1431: done (9 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1432: done (8 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1433: done (7 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1434: done (6 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1435: done (5 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1436: done (4 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1437: done (3 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1438: done (2 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1439: done (1 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1440: done (0 left) 2023-03-29T14:08:22.472 INFO:tasks.rados.rados.0.smithi017.stdout:1441: delete oid 47 current snap is 167 2023-03-29T14:08:22.551 INFO:tasks.rados.rados.0.smithi017.stdout:1441: done (0 left) 2023-03-29T14:08:22.551 INFO:tasks.rados.rados.0.smithi017.stdout:1442: write oid 27 current snap is 167 2023-03-29T14:08:22.551 INFO:tasks.rados.rados.0.smithi017.stdout:1442: seq_num 694 ranges {464058=410929,1394621=698906,2625204=592071,3656107=7951} 2023-03-29T14:08:22.555 INFO:tasks.rados.rados.0.smithi017.stdout:1442: writing smithi017107884-27 from 464058 to 874987 tid 1 2023-03-29T14:08:22.562 INFO:tasks.rados.rados.0.smithi017.stdout:1442: writing smithi017107884-27 from 1394621 to 2093527 tid 2 2023-03-29T14:08:22.568 INFO:tasks.rados.rados.0.smithi017.stdout:1442: writing smithi017107884-27 from 2625204 to 3217275 tid 3 2023-03-29T14:08:22.571 INFO:tasks.rados.rados.0.smithi017.stdout:1442: writing smithi017107884-27 from 3656107 to 3664058 tid 4 2023-03-29T14:08:22.571 INFO:tasks.rados.rados.0.smithi017.stdout:1443: delete oid 38 current snap is 167 2023-03-29T14:08:22.589 INFO:tasks.rados.rados.0.smithi017.stdout:1443: done (1 left) 2023-03-29T14:08:22.589 INFO:tasks.rados.rados.0.smithi017.stdout:1444: snap_remove snap 159 2023-03-29T14:08:22.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~193s 2023-03-29T14:08:23.249 INFO:tasks.rados.rados.0.smithi017.stdout:1444: done (1 left) 2023-03-29T14:08:23.249 INFO:tasks.rados.rados.0.smithi017.stdout:1445: write (excl) oid 42 current snap is 167 2023-03-29T14:08:23.249 INFO:tasks.rados.rados.0.smithi017.stdout:1442: finishing write tid 1 to smithi017107884-27 2023-03-29T14:08:23.249 INFO:tasks.rados.rados.0.smithi017.stdout:1445: seq_num 696 ranges {497672=561427,1795232=702575,2963697=730968} 2023-03-29T14:08:23.257 INFO:tasks.rados.rados.0.smithi017.stdout:1445: writing smithi017107884-42 from 497672 to 1059099 tid 1 2023-03-29T14:08:23.269 INFO:tasks.rados.rados.0.smithi017.stdout:1445: writing smithi017107884-42 from 1795232 to 2497807 tid 2 2023-03-29T14:08:23.279 INFO:tasks.rados.rados.0.smithi017.stdout:1445: writing smithi017107884-42 from 2963697 to 3694665 tid 3 2023-03-29T14:08:23.279 INFO:tasks.rados.rados.0.smithi017.stdout:1446: snap_remove snap 161 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1446: done (2 left) 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1447: write (excl) oid 44 current snap is 167 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1442: finishing write tid 2 to smithi017107884-27 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1442: finishing write tid 3 to smithi017107884-27 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1442: finishing write tid 4 to smithi017107884-27 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1442: finishing write tid 5 to smithi017107884-27 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1442: finishing write tid 6 to smithi017107884-27 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1442: oid 27 updating version 0 to 163 2023-03-29T14:08:24.254 INFO:tasks.rados.rados.0.smithi017.stdout:1442: oid 27 version 163 is already newer than 162 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1442: oid 27 version 163 is already newer than 163 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1442: oid 27 version 163 is already newer than 163 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1442: oid 27 version 163 is already newer than 163 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 163 (ObjNum 694 snap 167 seq_num 694) dirty exists 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1442: left oid 27 (ObjNum 694 snap 167 seq_num 694) 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1445: finishing write tid 1 to smithi017107884-42 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1445: finishing write tid 2 to smithi017107884-42 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1445: finishing write tid 3 to smithi017107884-42 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1445: finishing write tid 4 to smithi017107884-42 2023-03-29T14:08:24.255 INFO:tasks.rados.rados.0.smithi017.stdout:1447: seq_num 697 ranges {514479=636676,1853714=446233,2991120=542240} 2023-03-29T14:08:24.260 INFO:tasks.rados.rados.0.smithi017.stdout:1447: writing smithi017107884-44 from 514479 to 1151155 tid 1 2023-03-29T14:08:24.266 INFO:tasks.rados.rados.0.smithi017.stdout:1447: writing smithi017107884-44 from 1853714 to 2299947 tid 2 2023-03-29T14:08:24.273 INFO:tasks.rados.rados.0.smithi017.stdout:1447: writing smithi017107884-44 from 2991120 to 3533360 tid 3 2023-03-29T14:08:24.273 INFO:tasks.rados.rados.0.smithi017.stdout:1442: done (2 left) 2023-03-29T14:08:24.273 INFO:tasks.rados.rados.0.smithi017.stdout:1448: snap_create 2023-03-29T14:08:24.274 INFO:tasks.rados.rados.0.smithi017.stdout:1445: finishing write tid 5 to smithi017107884-42 2023-03-29T14:08:24.274 INFO:tasks.rados.rados.0.smithi017.stdout:1445: oid 42 updating version 0 to 218 2023-03-29T14:08:24.274 INFO:tasks.rados.rados.0.smithi017.stdout:1445: oid 42 version 218 is already newer than 214 2023-03-29T14:08:24.274 INFO:tasks.rados.rados.0.smithi017.stdout:1445: oid 42 version 218 is already newer than 218 2023-03-29T14:08:24.274 INFO:tasks.rados.rados.0.smithi017.stdout:1445: oid 42 version 218 is already newer than 218 2023-03-29T14:08:24.274 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 218 (ObjNum 696 snap 167 seq_num 696) dirty exists 2023-03-29T14:08:24.274 INFO:tasks.rados.rados.0.smithi017.stdout:1445: left oid 42 (ObjNum 696 snap 167 seq_num 696) 2023-03-29T14:08:24.409 INFO:tasks.rados.rados.0.smithi017.stdout:1447: finishing write tid 1 to smithi017107884-44 2023-03-29T14:08:24.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:24.438 INFO:tasks.rados.rados.0.smithi017.stdout:1447: finishing write tid 2 to smithi017107884-44 2023-03-29T14:08:24.480 INFO:tasks.rados.rados.0.smithi017.stdout:1447: finishing write tid 3 to smithi017107884-44 2023-03-29T14:08:24.481 INFO:tasks.rados.rados.0.smithi017.stdout:1447: finishing write tid 4 to smithi017107884-44 2023-03-29T14:08:24.483 INFO:tasks.rados.rados.0.smithi017.stdout:1447: finishing write tid 5 to smithi017107884-44 2023-03-29T14:08:24.483 INFO:tasks.rados.rados.0.smithi017.stdout:1447: oid 44 updating version 0 to 105 2023-03-29T14:08:24.484 INFO:tasks.rados.rados.0.smithi017.stdout:1447: oid 44 updating version 105 to 109 2023-03-29T14:08:24.484 INFO:tasks.rados.rados.0.smithi017.stdout:1447: oid 44 version 109 is already newer than 109 2023-03-29T14:08:24.484 INFO:tasks.rados.rados.0.smithi017.stdout:1447: oid 44 version 109 is already newer than 109 2023-03-29T14:08:24.484 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 109 (ObjNum 697 snap 167 seq_num 697) dirty exists 2023-03-29T14:08:24.484 INFO:tasks.rados.rados.0.smithi017.stdout:1447: left oid 44 (ObjNum 697 snap 167 seq_num 697) 2023-03-29T14:08:24.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:24.549+0000 7f9eb48f6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:24.554+0000 7f9eb48f6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:24.616 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:24.618 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:24.619 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1445: done (2 left) 2023-03-29T14:08:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1447: done (1 left) 2023-03-29T14:08:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1448: done (0 left) 2023-03-29T14:08:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1449: read oid 26 snap 164 2023-03-29T14:08:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1449: expect (ObjNum 615 snap 144 seq_num 615) 2023-03-29T14:08:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1450: write (excl) oid 44 current snap is 168 2023-03-29T14:08:25.256 INFO:tasks.rados.rados.0.smithi017.stdout:1450: seq_num 698 ranges {531286=711925,1912196=706244,3018543=712743} 2023-03-29T14:08:25.263 INFO:tasks.rados.rados.0.smithi017.stdout:1450: writing smithi017107884-44 from 531286 to 1243211 tid 1 2023-03-29T14:08:25.270 INFO:tasks.rados.rados.0.smithi017.stdout:1450: writing smithi017107884-44 from 1912196 to 2618440 tid 2 2023-03-29T14:08:25.277 INFO:tasks.rados.rados.0.smithi017.stdout:1450: writing smithi017107884-44 from 3018543 to 3731286 tid 3 2023-03-29T14:08:25.277 INFO:tasks.rados.rados.0.smithi017.stdout:1451: write oid 20 current snap is 168 2023-03-29T14:08:25.277 INFO:tasks.rados.rados.0.smithi017.stdout:1451: seq_num 699 ranges {548093=787174,1970678=449902,3045966=681137} 2023-03-29T14:08:25.284 INFO:tasks.rados.rados.0.smithi017.stdout:1451: writing smithi017107884-20 from 548093 to 1335267 tid 1 2023-03-29T14:08:25.290 INFO:tasks.rados.rados.0.smithi017.stdout:1451: writing smithi017107884-20 from 1970678 to 2420580 tid 2 2023-03-29T14:08:25.297 INFO:tasks.rados.rados.0.smithi017.stdout:1451: writing smithi017107884-20 from 3045966 to 3727103 tid 3 2023-03-29T14:08:25.297 INFO:tasks.rados.rados.0.smithi017.stdout:1452: delete oid 16 current snap is 168 2023-03-29T14:08:25.303 INFO:tasks.rados.rados.0.smithi017.stdout:1452: done (3 left) 2023-03-29T14:08:25.303 INFO:tasks.rados.rados.0.smithi017.stdout:1453: snap_create 2023-03-29T14:08:25.306 INFO:tasks.rados.rados.0.smithi017.stdout:1449: expect (ObjNum 615 snap 144 seq_num 615) 2023-03-29T14:08:25.428 INFO:tasks.rados.rados.0.smithi017.stdout:1450: finishing write tid 1 to smithi017107884-44 2023-03-29T14:08:25.468 INFO:tasks.rados.rados.0.smithi017.stdout:1451: finishing write tid 1 to smithi017107884-20 2023-03-29T14:08:25.483 INFO:tasks.rados.rados.0.smithi017.stdout:1451: finishing write tid 2 to smithi017107884-20 2023-03-29T14:08:25.485 INFO:tasks.rados.rados.0.smithi017.stdout:1450: finishing write tid 2 to smithi017107884-44 2023-03-29T14:08:25.488 INFO:tasks.rados.rados.0.smithi017.stdout:1451: finishing write tid 3 to smithi017107884-20 2023-03-29T14:08:25.501 INFO:tasks.rados.rados.0.smithi017.stdout:1450: finishing write tid 3 to smithi017107884-44 2023-03-29T14:08:25.503 INFO:tasks.rados.rados.0.smithi017.stdout:1450: finishing write tid 4 to smithi017107884-44 2023-03-29T14:08:25.506 INFO:tasks.rados.rados.0.smithi017.stdout:1450: finishing write tid 5 to smithi017107884-44 2023-03-29T14:08:25.507 INFO:tasks.rados.rados.0.smithi017.stdout:1450: oid 44 updating version 0 to 110 2023-03-29T14:08:25.507 INFO:tasks.rados.rados.0.smithi017.stdout:1450: oid 44 updating version 110 to 114 2023-03-29T14:08:25.507 INFO:tasks.rados.rados.0.smithi017.stdout:1450: oid 44 version 114 is already newer than 114 2023-03-29T14:08:25.507 INFO:tasks.rados.rados.0.smithi017.stdout:1450: oid 44 version 114 is already newer than 114 2023-03-29T14:08:25.507 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 114 (ObjNum 698 snap 168 seq_num 698) dirty exists 2023-03-29T14:08:25.507 INFO:tasks.rados.rados.0.smithi017.stdout:1450: left oid 44 (ObjNum 698 snap 168 seq_num 698) 2023-03-29T14:08:25.507 INFO:tasks.rados.rados.0.smithi017.stdout:1451: finishing write tid 4 to smithi017107884-20 2023-03-29T14:08:25.510 INFO:tasks.rados.rados.0.smithi017.stdout:1451: finishing write tid 5 to smithi017107884-20 2023-03-29T14:08:25.510 INFO:tasks.rados.rados.0.smithi017.stdout:1451: oid 20 updating version 0 to 198 2023-03-29T14:08:25.510 INFO:tasks.rados.rados.0.smithi017.stdout:1451: oid 20 version 198 is already newer than 198 2023-03-29T14:08:25.510 INFO:tasks.rados.rados.0.smithi017.stdout:1451: oid 20 version 198 is already newer than 197 2023-03-29T14:08:25.510 INFO:tasks.rados.rados.0.smithi017.stdout:1451: oid 20 version 198 is already newer than 198 2023-03-29T14:08:25.511 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 198 (ObjNum 699 snap 168 seq_num 699) dirty exists 2023-03-29T14:08:25.511 INFO:tasks.rados.rados.0.smithi017.stdout:1451: left oid 20 (ObjNum 699 snap 168 seq_num 699) 2023-03-29T14:08:25.888 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:25.888 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:25.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:26.019+0000 7fc0e117f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:26.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:26.025+0000 7fc0e117f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:26.233 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:26.245 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2023-03-29T14:08:26.168505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":218436664,"num_objects":776,"num_object_clones":680,"num_object_copies":2328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":3194,"num_read_kb":1843078,"num_write":4392,"num_write_kb":1197770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2316,"ondisk_log_size":2316,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1510972,"kb_used_data":1430044,"kb_used_omap":0,"kb_used_meta":77824,"kb_avail":375976388,"statfs":{"total":386547056640,"available":384999821312,"internally_reserved":0,"allocated":1464365056,"data_stored":1784400140,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":79691776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000879"},"pg_stats":[{"pgid":"3.f","version":"349'101","reported_seq":304,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:20.488549+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:20.488549+0000","last_peered":"2023-03-29T14:08:20.488549+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:20.488549+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6756678,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":108,"num_read_kb":65940,"num_write":116,"num_write_kb":33662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"344'39","reported_seq":356,"reported_epoch":352,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:18.507808+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:18.507808+0000","last_peered":"2023-03-29T14:08:18.507808+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:18.507808+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"354'231","reported_seq":810,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-03-29T14:08:21.524982+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:21.524982+0000","last_peered":"2023-03-29T14:08:21.524982+0000","last_clean":"2023-03-29T14:08:21.524982+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:21.524982+0000","last_undegraded":"2023-03-29T14:08:21.524982+0000","last_fullsized":"2023-03-29T14:08:21.524982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31295954,"num_objects":82,"num_object_clones":72,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":332,"num_read_kb":231785,"num_write":448,"num_write_kb":133106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"353'208","reported_seq":759,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-03-29T14:08:20.780684+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:20.780684+0000","last_peered":"2023-03-29T14:08:20.780684+0000","last_clean":"2023-03-29T14:08:20.780684+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:20.780684+0000","last_undegraded":"2023-03-29T14:08:20.780684+0000","last_fullsized":"2023-03-29T14:08:20.780684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20639788,"num_objects":82,"num_object_clones":74,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":301,"num_read_kb":184689,"num_write":406,"num_write_kb":114884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"349'193","reported_seq":693,"reported_epoch":352,"state":"active+clean","last_fresh":"2023-03-29T14:08:18.536890+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:18.536890+0000","last_peered":"2023-03-29T14:08:18.536890+0000","last_clean":"2023-03-29T14:08:18.536890+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:18.536890+0000","last_undegraded":"2023-03-29T14:08:18.536890+0000","last_fullsized":"2023-03-29T14:08:18.536890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12471322,"num_objects":70,"num_object_clones":62,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":242,"num_read_kb":136349,"num_write":382,"num_write_kb":111528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":296,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-03-29T14:08:20.485071+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:20.485071+0000","last_peered":"2023-03-29T14:08:20.485071+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:20.485071+0000","last_undegraded":"2023-03-29T14:08:20.485071+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"315'115","reported_seq":542,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:21.539646+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:21.539646+0000","last_peered":"2023-03-29T14:08:21.539646+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:21.539646+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4975136,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":144,"num_read_kb":67874,"num_write":208,"num_write_kb":60502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":42}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":470,"reported_epoch":354,"state":"active+clean","last_fresh":"2023-03-29T14:08:20.478489+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:20.478489+0000","last_peered":"2023-03-29T14:08:20.478489+0000","last_clean":"2023-03-29T14:08:20.478489+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:20.478489+0000","last_undegraded":"2023-03-29T14:08:20.478489+0000","last_fullsized":"2023-03-29T14:08:20.478489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":277,"reported_epoch":352,"state":"active+undersized","last_fresh":"2023-03-29T14:08:18.512363+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:18.512363+0000","last_peered":"2023-03-29T14:08:18.512363+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:18.512363+0000","last_undegraded":"2023-03-29T14:08:18.512363+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"334'86","reported_seq":240,"reported_epoch":353,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:20.480313+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:20.480313+0000","last_peered":"2023-03-29T14:08:20.480313+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:20.480313+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050784,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":12593,"num_write":96,"num_write_kb":25750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":299,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-03-29T14:08:20.495876+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:20.495876+0000","last_peered":"2023-03-29T14:08:20.495876+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:20.495876+0000","last_undegraded":"2023-03-29T14:08:20.495876+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"334'88","reported_seq":486,"reported_epoch":353,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:19.724084+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:19.724084+0000","last_peered":"2023-03-29T14:08:19.724084+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:19.724084+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":100905,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":352,"state":"active+undersized","last_fresh":"2023-03-29T14:08:18.518025+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:18.518025+0000","last_peered":"2023-03-29T14:08:18.518025+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:18.518025+0000","last_undegraded":"2023-03-29T14:08:18.518025+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"349'124","reported_seq":551,"reported_epoch":352,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:20.097765+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:20.097765+0000","last_peered":"2023-03-29T14:08:20.097765+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:20.097765+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17480072,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":90467,"num_write":242,"num_write_kb":71066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":299,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-03-29T14:08:20.493095+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:20.493095+0000","last_peered":"2023-03-29T14:08:20.493095+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:20.493095+0000","last_undegraded":"2023-03-29T14:08:20.493095+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":502,"reported_epoch":352,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:18.532360+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:18.532360+0000","last_peered":"2023-03-29T14:08:18.532360+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:18.532360+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"352'156","reported_seq":692,"reported_epoch":354,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:20.510037+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:20.510037+0000","last_peered":"2023-03-29T14:08:20.510037+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:20.510037+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11563744,"num_objects":56,"num_object_clones":48,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":311,"num_read_kb":211483,"num_write":302,"num_write_kb":87280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":56}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":299,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-03-29T14:08:20.479587+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:20.479587+0000","last_peered":"2023-03-29T14:08:20.479587+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:20.479587+0000","last_undegraded":"2023-03-29T14:08:20.479587+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"352'299","reported_seq":957,"reported_epoch":353,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:19.883565+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:19.883565+0000","last_peered":"2023-03-29T14:08:19.883565+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:19.883565+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31464024,"num_objects":104,"num_object_clones":90,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":453,"num_read_kb":272843,"num_write":560,"num_write_kb":161574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":352,"state":"active+undersized","last_fresh":"2023-03-29T14:08:18.542981+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:18.542981+0000","last_peered":"2023-03-29T14:08:18.542981+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:18.542981+0000","last_undegraded":"2023-03-29T14:08:18.542981+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"352'104","reported_seq":500,"reported_epoch":352,"state":"active+clean","last_fresh":"2023-03-29T14:08:19.661153+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:19.661153+0000","last_peered":"2023-03-29T14:08:19.661153+0000","last_clean":"2023-03-29T14:08:19.661153+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:19.661153+0000","last_undegraded":"2023-03-29T14:08:19.661153+0000","last_fullsized":"2023-03-29T14:08:19.661153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13177216,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":70182,"num_write":214,"num_write_kb":60716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"349'195","reported_seq":696,"reported_epoch":352,"state":"active+clean","last_fresh":"2023-03-29T14:08:19.723245+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:19.723245+0000","last_peered":"2023-03-29T14:08:19.723245+0000","last_clean":"2023-03-29T14:08:19.723245+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:19.723245+0000","last_undegraded":"2023-03-29T14:08:19.723245+0000","last_fullsized":"2023-03-29T14:08:19.723245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20101832,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":257,"num_read_kb":159204,"num_write":360,"num_write_kb":104988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":277,"reported_epoch":352,"state":"active+undersized","last_fresh":"2023-03-29T14:08:18.525365+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:18.525365+0000","last_peered":"2023-03-29T14:08:18.525365+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:18.525365+0000","last_undegraded":"2023-03-29T14:08:18.525365+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":299,"reported_epoch":354,"state":"active+undersized","last_fresh":"2023-03-29T14:08:20.496793+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:20.496793+0000","last_peered":"2023-03-29T14:08:20.496793+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:20.496793+0000","last_undegraded":"2023-03-29T14:08:20.496793+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"352'201","reported_seq":722,"reported_epoch":353,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:20.152332+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:20.152332+0000","last_peered":"2023-03-29T14:08:20.152332+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:20.152332+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28942578,"num_objects":70,"num_object_clones":62,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":155246,"num_write":388,"num_write_kb":114584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":217255928,"num_objects":772,"num_object_clones":680,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":3052,"num_read_kb":1842958,"num_write":4118,"num_write_kb":1193982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2219,"ondisk_log_size":2219,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706039,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":432760,"kb_used_data":410040,"kb_used_omap":0,"kb_used_meta":21376,"kb_avail":93939080,"statfs":{"total":96636764160,"available":96193617920,"internally_reserved":0,"allocated":419880960,"data_stored":511268584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":21889024},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":486472,"kb_used_data":463472,"kb_used_omap":0,"kb_used_meta":22592,"kb_avail":93885368,"statfs":{"total":96636764160,"available":96138616832,"internally_reserved":0,"allocated":474595328,"data_stored":583731067,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23134208},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":435796,"kb_used_data":412692,"kb_used_omap":0,"kb_used_meta":21760,"kb_avail":93936044,"statfs":{"total":96636764160,"available":96190509056,"internally_reserved":0,"allocated":422596608,"data_stored":516168264,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22282240},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:26.248 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.9 3.5 3.6 2023-03-29T14:08:26.259 INFO:tasks.rados.rados.0.smithi017.stdout:1449: done (3 left) 2023-03-29T14:08:26.259 INFO:tasks.rados.rados.0.smithi017.stdout:1450: done (2 left) 2023-03-29T14:08:26.260 INFO:tasks.rados.rados.0.smithi017.stdout:1451: done (1 left) 2023-03-29T14:08:26.260 INFO:tasks.rados.rados.0.smithi017.stdout:1453: done (0 left) 2023-03-29T14:08:26.260 INFO:tasks.rados.rados.0.smithi017.stdout:1454: snap_remove snap 167 2023-03-29T14:08:26.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:26.378+0000 7fd4f9ebd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:26.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:26.388+0000 7fd4f9ebd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:26.597 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.5,3.6] on osd.0 to force-recovery; instructing pg(s) [3.9] on osd.3 to force-recovery; 2023-03-29T14:08:26.597 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:08:26.620 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:26.778 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:26.777+0000 7ff4b687a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:26.781 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:26.781+0000 7ff4b687a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:26.845 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:26.847 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:26.848 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:27.261 INFO:tasks.rados.rados.0.smithi017.stdout:1454: done (0 left) 2023-03-29T14:08:27.261 INFO:tasks.rados.rados.0.smithi017.stdout:1455: read oid 36 snap -1 2023-03-29T14:08:27.261 INFO:tasks.rados.rados.0.smithi017.stdout:1455: expect (ObjNum 671 snap 164 seq_num 671) 2023-03-29T14:08:27.261 INFO:tasks.rados.rados.0.smithi017.stdout:1456: snap_create 2023-03-29T14:08:27.356 INFO:tasks.rados.rados.0.smithi017.stdout:1455: expect (ObjNum 671 snap 164 seq_num 671) 2023-03-29T14:08:28.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~198s 2023-03-29T14:08:28.263 INFO:tasks.rados.rados.0.smithi017.stdout:1455: done (1 left) 2023-03-29T14:08:28.264 INFO:tasks.rados.rados.0.smithi017.stdout:1456: done (0 left) 2023-03-29T14:08:28.264 INFO:tasks.rados.rados.0.smithi017.stdout:1457: write (excl) oid 38 current snap is 170 2023-03-29T14:08:28.264 INFO:tasks.rados.rados.0.smithi017.stdout:1457: seq_num 701 ranges {581707=654025,1687642=453571,2700812=703681} 2023-03-29T14:08:28.270 INFO:tasks.rados.rados.0.smithi017.stdout:1457: writing smithi017107884-38 from 581707 to 1235732 tid 1 2023-03-29T14:08:28.275 INFO:tasks.rados.rados.0.smithi017.stdout:1457: writing smithi017107884-38 from 1687642 to 2141213 tid 2 2023-03-29T14:08:28.282 INFO:tasks.rados.rados.0.smithi017.stdout:1457: writing smithi017107884-38 from 2700812 to 3404493 tid 3 2023-03-29T14:08:28.282 INFO:tasks.rados.rados.0.smithi017.stdout:1458: delete oid 12 current snap is 170 2023-03-29T14:08:28.376 INFO:tasks.rados.rados.0.smithi017.stdout:1457: finishing write tid 1 to smithi017107884-38 2023-03-29T14:08:28.412 INFO:tasks.rados.rados.0.smithi017.stdout:1458: done (1 left) 2023-03-29T14:08:28.412 INFO:tasks.rados.rados.0.smithi017.stdout:1459: write oid 36 current snap is 170 2023-03-29T14:08:28.412 INFO:tasks.rados.rados.0.smithi017.stdout:1459: seq_num 703 ranges {615321=404523,1804606=457240,2755658=726225} 2023-03-29T14:08:28.417 INFO:tasks.rados.rados.0.smithi017.stdout:1459: writing smithi017107884-36 from 615321 to 1019844 tid 1 2023-03-29T14:08:28.424 INFO:tasks.rados.rados.0.smithi017.stdout:1459: writing smithi017107884-36 from 1804606 to 2261846 tid 2 2023-03-29T14:08:28.431 INFO:tasks.rados.rados.0.smithi017.stdout:1459: writing smithi017107884-36 from 2755658 to 3481883 tid 3 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1460: read oid 17 snap -1 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1460: expect (ObjNum 659 snap 160 seq_num 659) 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1461: read oid 40 snap 169 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1461: expect (ObjNum 685 snap 166 seq_num 685) 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1462: read oid 16 snap 164 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1462: expect deleted 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1463: write oid 27 current snap is 170 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1457: finishing write tid 2 to smithi017107884-38 2023-03-29T14:08:28.432 INFO:tasks.rados.rados.0.smithi017.stdout:1463: seq_num 704 ranges {632128=479772,1746735=600898,3066728=653850} 2023-03-29T14:08:28.437 INFO:tasks.rados.rados.0.smithi017.stdout:1463: writing smithi017107884-27 from 632128 to 1111900 tid 1 2023-03-29T14:08:28.444 INFO:tasks.rados.rados.0.smithi017.stdout:1463: writing smithi017107884-27 from 1746735 to 2347633 tid 2 2023-03-29T14:08:28.452 INFO:tasks.rados.rados.0.smithi017.stdout:1463: writing smithi017107884-27 from 3066728 to 3720578 tid 3 2023-03-29T14:08:28.452 INFO:tasks.rados.rados.0.smithi017.stdout:1464: write oid 22 current snap is 170 2023-03-29T14:08:28.452 INFO:tasks.rados.rados.0.smithi017.stdout:1457: finishing write tid 3 to smithi017107884-38 2023-03-29T14:08:28.452 INFO:tasks.rados.rados.0.smithi017.stdout:1457: finishing write tid 4 to smithi017107884-38 2023-03-29T14:08:28.452 INFO:tasks.rados.rados.0.smithi017.stdout:1464: seq_num 705 ranges {648935=555021,1805217=460909,2694151=465122,3817442=31493} 2023-03-29T14:08:28.458 INFO:tasks.rados.rados.0.smithi017.stdout:1464: writing smithi017107884-22 from 648935 to 1203956 tid 1 2023-03-29T14:08:28.464 INFO:tasks.rados.rados.0.smithi017.stdout:1464: writing smithi017107884-22 from 1805217 to 2266126 tid 2 2023-03-29T14:08:28.469 INFO:tasks.rados.rados.0.smithi017.stdout:1464: writing smithi017107884-22 from 2694151 to 3159273 tid 3 2023-03-29T14:08:28.473 INFO:tasks.rados.rados.0.smithi017.stdout:1464: writing smithi017107884-22 from 3817442 to 3848935 tid 4 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1465: read oid 9 snap -1 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1465: expect (ObjNum 691 snap 166 seq_num 691) 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1466: write (excl) oid 29 current snap is 170 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1457: finishing write tid 5 to smithi017107884-38 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1457: oid 38 updating version 0 to 41 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1457: oid 38 updating version 41 to 44 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1457: oid 38 version 44 is already newer than 44 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1457: oid 38 version 44 is already newer than 44 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 44 (ObjNum 701 snap 170 seq_num 701) dirty exists 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1457: left oid 38 (ObjNum 701 snap 170 seq_num 701) 2023-03-29T14:08:28.474 INFO:tasks.rados.rados.0.smithi017.stdout:1466: seq_num 706 ranges {665742=630270,1863699=604567,3121574=676394} 2023-03-29T14:08:28.480 INFO:tasks.rados.rados.0.smithi017.stdout:1466: writing smithi017107884-29 from 665742 to 1296012 tid 1 2023-03-29T14:08:28.487 INFO:tasks.rados.rados.0.smithi017.stdout:1466: writing smithi017107884-29 from 1863699 to 2468266 tid 2 2023-03-29T14:08:28.495 INFO:tasks.rados.rados.0.smithi017.stdout:1466: writing smithi017107884-29 from 3121574 to 3797968 tid 3 2023-03-29T14:08:28.495 INFO:tasks.rados.rados.0.smithi017.stdout:1457: done (8 left) 2023-03-29T14:08:28.496 INFO:tasks.rados.rados.0.smithi017.stdout:1467: snap_create 2023-03-29T14:08:28.496 INFO:tasks.rados.rados.0.smithi017.stdout:1459: finishing write tid 1 to smithi017107884-36 2023-03-29T14:08:28.496 INFO:tasks.rados.rados.0.smithi017.stdout:1459: finishing write tid 2 to smithi017107884-36 2023-03-29T14:08:28.496 INFO:tasks.rados.rados.0.smithi017.stdout:1460: expect (ObjNum 659 snap 160 seq_num 659) 2023-03-29T14:08:28.653 INFO:tasks.rados.rados.0.smithi017.stdout:1461: expect (ObjNum 685 snap 166 seq_num 685) 2023-03-29T14:08:28.848 INFO:tasks.rados.rados.0.smithi017.stdout:1459: finishing write tid 3 to smithi017107884-36 2023-03-29T14:08:28.848 INFO:tasks.rados.rados.0.smithi017.stdout:1459: finishing write tid 4 to smithi017107884-36 2023-03-29T14:08:28.848 INFO:tasks.rados.rados.0.smithi017.stdout:1459: finishing write tid 5 to smithi017107884-36 2023-03-29T14:08:28.848 INFO:tasks.rados.rados.0.smithi017.stdout:1459: oid 36 updating version 0 to 128 2023-03-29T14:08:28.848 INFO:tasks.rados.rados.0.smithi017.stdout:1459: oid 36 version 128 is already newer than 127 2023-03-29T14:08:28.848 INFO:tasks.rados.rados.0.smithi017.stdout:1459: oid 36 updating version 128 to 129 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:1459: oid 36 version 129 is already newer than 129 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 129 (ObjNum 703 snap 170 seq_num 703) dirty exists 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:1459: left oid 36 (ObjNum 703 snap 170 seq_num 703) 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:1463: finishing write tid 1 to smithi017107884-27 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:1463: finishing write tid 2 to smithi017107884-27 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:1463: finishing write tid 3 to smithi017107884-27 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:1463: finishing write tid 4 to smithi017107884-27 2023-03-29T14:08:28.849 INFO:tasks.rados.rados.0.smithi017.stdout:1465: expect (ObjNum 691 snap 166 seq_num 691) 2023-03-29T14:08:28.849 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:28.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:28.978+0000 7f1cf84d8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:28.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:28.983+0000 7f1cf84d8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:29.044 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:29.048 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:29.048 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:29.111 INFO:tasks.rados.rados.0.smithi017.stdout:1463: finishing write tid 5 to smithi017107884-27 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:1463: oid 27 updating version 0 to 168 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:1463: oid 27 version 168 is already newer than 168 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:1463: oid 27 version 168 is already newer than 168 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:1463: oid 27 version 168 is already newer than 168 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 168 (ObjNum 704 snap 170 seq_num 704) dirty exists 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:1463: left oid 27 (ObjNum 704 snap 170 seq_num 704) 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:1464: finishing write tid 1 to smithi017107884-22 2023-03-29T14:08:29.112 INFO:tasks.rados.rados.0.smithi017.stdout:1466: finishing write tid 1 to smithi017107884-29 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: finishing write tid 2 to smithi017107884-29 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: finishing write tid 3 to smithi017107884-29 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: finishing write tid 4 to smithi017107884-29 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: finishing write tid 5 to smithi017107884-29 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: oid 29 updating version 0 to 93 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: oid 29 version 93 is already newer than 93 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: oid 29 version 93 is already newer than 89 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:1466: oid 29 version 93 is already newer than 93 2023-03-29T14:08:29.113 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 93 (ObjNum 706 snap 170 seq_num 706) dirty exists 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1466: left oid 29 (ObjNum 706 snap 170 seq_num 706) 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: finishing write tid 2 to smithi017107884-22 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: finishing write tid 3 to smithi017107884-22 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: finishing write tid 4 to smithi017107884-22 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: finishing write tid 5 to smithi017107884-22 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: finishing write tid 6 to smithi017107884-22 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: oid 22 updating version 0 to 108 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: oid 22 version 108 is already newer than 108 2023-03-29T14:08:29.114 INFO:tasks.rados.rados.0.smithi017.stdout:1464: oid 22 version 108 is already newer than 108 2023-03-29T14:08:29.115 INFO:tasks.rados.rados.0.smithi017.stdout:1464: oid 22 version 108 is already newer than 108 2023-03-29T14:08:29.115 INFO:tasks.rados.rados.0.smithi017.stdout:1464: oid 22 version 108 is already newer than 108 2023-03-29T14:08:29.115 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 108 (ObjNum 705 snap 170 seq_num 705) dirty exists 2023-03-29T14:08:29.115 INFO:tasks.rados.rados.0.smithi017.stdout:1464: left oid 22 (ObjNum 705 snap 170 seq_num 705) 2023-03-29T14:08:29.269 INFO:tasks.rados.rados.0.smithi017.stdout:1459: done (8 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1460: done (7 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1461: done (6 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1462: done (5 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1463: done (4 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1464: done (3 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1465: done (2 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1466: done (1 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1467: done (0 left) 2023-03-29T14:08:29.270 INFO:tasks.rados.rados.0.smithi017.stdout:1468: snap_create 2023-03-29T14:08:30.273 INFO:tasks.rados.rados.0.smithi017.stdout:1468: done (0 left) 2023-03-29T14:08:30.273 INFO:tasks.rados.rados.0.smithi017.stdout:1469: write (excl) oid 47 current snap is 172 2023-03-29T14:08:30.273 INFO:tasks.rados.rados.0.smithi017.stdout:1469: seq_num 707 ranges {682549=705519,1922181=464578,3148997=487666} 2023-03-29T14:08:30.280 INFO:tasks.rados.rados.0.smithi017.stdout:1469: writing smithi017107884-47 from 682549 to 1388068 tid 1 2023-03-29T14:08:30.286 INFO:tasks.rados.rados.0.smithi017.stdout:1469: writing smithi017107884-47 from 1922181 to 2386759 tid 2 2023-03-29T14:08:30.293 INFO:tasks.rados.rados.0.smithi017.stdout:1469: writing smithi017107884-47 from 3148997 to 3636663 tid 3 2023-03-29T14:08:30.293 INFO:tasks.rados.rados.0.smithi017.stdout:1470: read oid 11 snap 169 2023-03-29T14:08:30.293 INFO:tasks.rados.rados.0.smithi017.stdout:1470: expect (ObjNum 674 snap 164 seq_num 674) 2023-03-29T14:08:30.293 INFO:tasks.rados.rados.0.smithi017.stdout:1471: write (excl) oid 42 current snap is 172 2023-03-29T14:08:30.293 INFO:tasks.rados.rados.0.smithi017.stdout:1471: seq_num 708 ranges {699356=497121,1980663=608236,3176420=415291} 2023-03-29T14:08:30.300 INFO:tasks.rados.rados.0.smithi017.stdout:1471: writing smithi017107884-42 from 699356 to 1196477 tid 1 2023-03-29T14:08:30.308 INFO:tasks.rados.rados.0.smithi017.stdout:1471: writing smithi017107884-42 from 1980663 to 2588899 tid 2 2023-03-29T14:08:30.314 INFO:tasks.rados.rados.0.smithi017.stdout:1471: writing smithi017107884-42 from 3176420 to 3591711 tid 3 2023-03-29T14:08:30.314 INFO:tasks.rados.rados.0.smithi017.stdout:1472: snap_create 2023-03-29T14:08:30.423 INFO:tasks.rados.rados.0.smithi017.stdout:1469: finishing write tid 1 to smithi017107884-47 2023-03-29T14:08:30.427 INFO:tasks.rados.rados.0.smithi017.stdout:1471: finishing write tid 1 to smithi017107884-42 2023-03-29T14:08:30.434 INFO:tasks.rados.rados.0.smithi017.stdout:1470: expect (ObjNum 674 snap 164 seq_num 674) 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1471: finishing write tid 2 to smithi017107884-42 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1471: finishing write tid 3 to smithi017107884-42 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: finishing write tid 2 to smithi017107884-47 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: finishing write tid 3 to smithi017107884-47 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: finishing write tid 4 to smithi017107884-47 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: finishing write tid 5 to smithi017107884-47 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: oid 47 updating version 0 to 202 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: oid 47 version 202 is already newer than 202 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: oid 47 version 202 is already newer than 199 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: oid 47 version 202 is already newer than 202 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 202 (ObjNum 707 snap 172 seq_num 707) dirty exists 2023-03-29T14:08:30.569 INFO:tasks.rados.rados.0.smithi017.stdout:1469: left oid 47 (ObjNum 707 snap 172 seq_num 707) 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:1471: finishing write tid 4 to smithi017107884-42 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:1471: finishing write tid 5 to smithi017107884-42 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:1471: oid 42 updating version 0 to 223 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:1471: oid 42 version 223 is already newer than 223 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:1471: oid 42 version 223 is already newer than 219 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:1471: oid 42 version 223 is already newer than 223 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 223 (ObjNum 708 snap 172 seq_num 708) dirty exists 2023-03-29T14:08:30.570 INFO:tasks.rados.rados.0.smithi017.stdout:1471: left oid 42 (ObjNum 708 snap 172 seq_num 708) 2023-03-29T14:08:31.049 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:31.178 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:31.177+0000 7fb8f4a0c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:31.181 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:31.181+0000 7fb8f4a0c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:31.243 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:31.246 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:31.246 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1469: done (3 left) 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1470: done (2 left) 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1471: done (1 left) 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1472: done (0 left) 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1473: read oid 14 snap -1 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1473: expect deleted 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1474: read oid 23 snap -1 2023-03-29T14:08:31.274 INFO:tasks.rados.rados.0.smithi017.stdout:1474: expect deleted 2023-03-29T14:08:31.275 INFO:tasks.rados.rados.0.smithi017.stdout:1475: snap_create 2023-03-29T14:08:31.609 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:31.609 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:31.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:31.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:31.739+0000 7fc1314b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:31.745+0000 7fc1314b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:32.162 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:32.182 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":550,"stamp":"2023-03-29T14:08:31.272695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227680514,"num_objects":790,"num_object_clones":694,"num_object_copies":2370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":3249,"num_read_kb":1881062,"num_write":4470,"num_write_kb":1224470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2358,"ondisk_log_size":2358,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1555008,"kb_used_data":1476448,"kb_used_omap":0,"kb_used_meta":78528,"kb_avail":375932352,"statfs":{"total":386547056640,"available":384954728448,"internally_reserved":0,"allocated":1511882752,"data_stored":1846999853,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":80412672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013480"},"pg_stats":[{"pgid":"3.f","version":"349'101","reported_seq":309,"reported_epoch":359,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:25.302513+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:25.302513+0000","last_peered":"2023-03-29T14:08:25.302513+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:25.302513+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6756678,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":108,"num_read_kb":65940,"num_write":116,"num_write_kb":33662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"344'39","reported_seq":361,"reported_epoch":358,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:24.350682+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:24.350682+0000","last_peered":"2023-03-29T14:08:24.350682+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:24.350682+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":48,"num_read_kb":25431,"num_write":72,"num_write_kb":21728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"354'232","reported_seq":815,"reported_epoch":359,"state":"active+clean","last_fresh":"2023-03-29T14:08:25.279841+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:25.279841+0000","last_peered":"2023-03-29T14:08:25.279841+0000","last_clean":"2023-03-29T14:08:25.279841+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:25.279841+0000","last_undegraded":"2023-03-29T14:08:25.279841+0000","last_fullsized":"2023-03-29T14:08:25.279841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31295954,"num_objects":82,"num_object_clones":72,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":231786,"num_write":448,"num_write_kb":133106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"356'218","reported_seq":781,"reported_epoch":359,"state":"active+clean","last_fresh":"2023-03-29T14:08:25.289015+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:25.289015+0000","last_peered":"2023-03-29T14:08:25.289015+0000","last_clean":"2023-03-29T14:08:25.289015+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:25.289015+0000","last_undegraded":"2023-03-29T14:08:25.289015+0000","last_fullsized":"2023-03-29T14:08:25.289015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21648208,"num_objects":86,"num_object_clones":78,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":307,"num_read_kb":184691,"num_write":426,"num_write_kb":122034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"358'198","reported_seq":709,"reported_epoch":358,"state":"active+clean","last_fresh":"2023-03-29T14:08:25.476474+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:25.476474+0000","last_peered":"2023-03-29T14:08:25.476474+0000","last_clean":"2023-03-29T14:08:25.476474+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:25.476474+0000","last_undegraded":"2023-03-29T14:08:25.476474+0000","last_fullsized":"2023-03-29T14:08:25.476474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13378900,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":250,"num_read_kb":145260,"num_write":392,"num_write_kb":115278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":300,"reported_epoch":359,"state":"active+undersized","last_fresh":"2023-03-29T14:08:25.283071+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:25.283071+0000","last_peered":"2023-03-29T14:08:25.283071+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:25.283071+0000","last_undegraded":"2023-03-29T14:08:25.283071+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"354'119","reported_seq":551,"reported_epoch":359,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:25.283841+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:25.283841+0000","last_peered":"2023-03-29T14:08:25.283841+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:25.283841+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193660,"num_objects":42,"num_object_clones":36,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":145,"num_read_kb":67875,"num_write":214,"num_write_kb":61854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":42}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":474,"reported_epoch":359,"state":"active+clean","last_fresh":"2023-03-29T14:08:25.279124+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:25.279124+0000","last_peered":"2023-03-29T14:08:25.279124+0000","last_clean":"2023-03-29T14:08:25.279124+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:25.279124+0000","last_undegraded":"2023-03-29T14:08:25.279124+0000","last_fullsized":"2023-03-29T14:08:25.279124+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":281,"reported_epoch":358,"state":"active+undersized","last_fresh":"2023-03-29T14:08:24.316124+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:24.316124+0000","last_peered":"2023-03-29T14:08:24.316124+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:24.316124+0000","last_undegraded":"2023-03-29T14:08:24.316124+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"334'86","reported_seq":245,"reported_epoch":359,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:24.283825+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:24.283825+0000","last_peered":"2023-03-29T14:08:24.283825+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:24.283825+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050784,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":12593,"num_write":96,"num_write_kb":25750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":303,"reported_epoch":359,"state":"active+undersized","last_fresh":"2023-03-29T14:08:25.291027+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:25.291027+0000","last_peered":"2023-03-29T14:08:25.291027+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:25.291027+0000","last_undegraded":"2023-03-29T14:08:25.291027+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"334'88","reported_seq":491,"reported_epoch":359,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:24.310548+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:24.310548+0000","last_peered":"2023-03-29T14:08:24.310548+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:24.310548+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4605382,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":146,"num_read_kb":100905,"num_write":172,"num_write_kb":49158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":358,"state":"active+undersized","last_fresh":"2023-03-29T14:08:24.352556+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:24.352556+0000","last_peered":"2023-03-29T14:08:24.352556+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:24.352556+0000","last_undegraded":"2023-03-29T14:08:24.352556+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"349'124","reported_seq":561,"reported_epoch":358,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:24.329368+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:24.329368+0000","last_peered":"2023-03-29T14:08:24.329368+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:24.329368+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17480072,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":185,"num_read_kb":105840,"num_write":242,"num_write_kb":71066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":303,"reported_epoch":359,"state":"active+undersized","last_fresh":"2023-03-29T14:08:25.290143+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:25.290143+0000","last_peered":"2023-03-29T14:08:25.290143+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:25.290143+0000","last_undegraded":"2023-03-29T14:08:25.290143+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":506,"reported_epoch":358,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:24.349358+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:24.349358+0000","last_peered":"2023-03-29T14:08:24.349358+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:24.349358+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"355'163","reported_seq":710,"reported_epoch":359,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:25.305738+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:25.305738+0000","last_peered":"2023-03-29T14:08:25.305738+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:25.305738+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13849496,"num_objects":58,"num_object_clones":50,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":319,"num_read_kb":215754,"num_write":314,"num_write_kb":90624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":58}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":303,"reported_epoch":359,"state":"active+undersized","last_fresh":"2023-03-29T14:08:25.286985+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:25.286985+0000","last_peered":"2023-03-29T14:08:25.286985+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:25.286985+0000","last_undegraded":"2023-03-29T14:08:25.286985+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"354'304","reported_seq":971,"reported_epoch":359,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:24.286844+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:24.286844+0000","last_peered":"2023-03-29T14:08:24.286844+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:24.286844+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33178338,"num_objects":106,"num_object_clones":92,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":456,"num_read_kb":272844,"num_write":570,"num_write_kb":165336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":106}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":358,"state":"active+undersized","last_fresh":"2023-03-29T14:08:24.306278+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:24.306278+0000","last_peered":"2023-03-29T14:08:24.306278+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:24.306278+0000","last_undegraded":"2023-03-29T14:08:24.306278+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"358'114","reported_seq":522,"reported_epoch":358,"state":"active+clean","last_fresh":"2023-03-29T14:08:25.469369+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:25.469369+0000","last_peered":"2023-03-29T14:08:25.469369+0000","last_clean":"2023-03-29T14:08:25.469369+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:25.469369+0000","last_undegraded":"2023-03-29T14:08:25.469369+0000","last_fullsized":"2023-03-29T14:08:25.469369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286478,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":70184,"num_write":234,"num_write_kb":68058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"349'195","reported_seq":701,"reported_epoch":358,"state":"active+clean","last_fresh":"2023-03-29T14:08:24.372317+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:24.372317+0000","last_peered":"2023-03-29T14:08:24.372317+0000","last_clean":"2023-03-29T14:08:24.372317+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:24.372317+0000","last_undegraded":"2023-03-29T14:08:24.372317+0000","last_fullsized":"2023-03-29T14:08:24.372317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20101832,"num_objects":68,"num_object_clones":58,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":258,"num_read_kb":159204,"num_write":360,"num_write_kb":104988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":281,"reported_epoch":358,"state":"active+undersized","last_fresh":"2023-03-29T14:08:24.358419+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:24.358419+0000","last_peered":"2023-03-29T14:08:24.358419+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:24.358419+0000","last_undegraded":"2023-03-29T14:08:24.358419+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":303,"reported_epoch":359,"state":"active+undersized","last_fresh":"2023-03-29T14:08:25.276262+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:25.276262+0000","last_peered":"2023-03-29T14:08:25.276262+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:25.276262+0000","last_undegraded":"2023-03-29T14:08:25.276262+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"352'201","reported_seq":730,"reported_epoch":359,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:24.304064+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:24.304064+0000","last_peered":"2023-03-29T14:08:24.304064+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:24.304064+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28942578,"num_objects":70,"num_object_clones":62,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":164668,"num_write":388,"num_write_kb":114584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":226499778,"num_objects":786,"num_object_clones":694,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":3107,"num_read_kb":1880942,"num_write":4196,"num_write_kb":1220682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2261,"ondisk_log_size":2261,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":447680,"kb_used_data":426104,"kb_used_omap":0,"kb_used_meta":21568,"kb_avail":93924160,"statfs":{"total":96636764160,"available":96178339840,"internally_reserved":0,"allocated":436330496,"data_stored":533523788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22085632},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":500884,"kb_used_data":478028,"kb_used_omap":0,"kb_used_meta":22848,"kb_avail":93870956,"statfs":{"total":96636764160,"available":96123858944,"internally_reserved":0,"allocated":489500672,"data_stored":602493964,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23396352},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":450500,"kb_used_data":428476,"kb_used_omap":0,"kb_used_meta":22016,"kb_avail":93921340,"statfs":{"total":96636764160,"available":96175452160,"internally_reserved":0,"allocated":438759424,"data_stored":537749876,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22544384},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:32.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.3 1.0 3.2 3.6 2023-03-29T14:08:32.284 INFO:tasks.rados.rados.0.smithi017.stdout:1473: done (2 left) 2023-03-29T14:08:32.284 INFO:tasks.rados.rados.0.smithi017.stdout:1474: done (1 left) 2023-03-29T14:08:32.284 INFO:tasks.rados.rados.0.smithi017.stdout:1475: done (0 left) 2023-03-29T14:08:32.284 INFO:tasks.rados.rados.0.smithi017.stdout:1476: write oid 17 current snap is 174 2023-03-29T14:08:32.284 INFO:tasks.rados.rados.0.smithi017.stdout:1476: seq_num 709 ranges {716163=572370,2039145=468247,3203843=626563} 2023-03-29T14:08:32.291 INFO:tasks.rados.rados.0.smithi017.stdout:1476: writing smithi017107884-17 from 716163 to 1288533 tid 1 2023-03-29T14:08:32.298 INFO:tasks.rados.rados.0.smithi017.stdout:1476: writing smithi017107884-17 from 2039145 to 2507392 tid 2 2023-03-29T14:08:32.305 INFO:tasks.rados.rados.0.smithi017.stdout:1476: writing smithi017107884-17 from 3203843 to 3830406 tid 3 2023-03-29T14:08:32.305 INFO:tasks.rados.rados.0.smithi017.stdout:1477: snap_create 2023-03-29T14:08:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:32.317+0000 7f266dc2e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:32.330+0000 7f266dc2e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:32.389 INFO:tasks.rados.rados.0.smithi017.stdout:1476: finishing write tid 1 to smithi017107884-17 2023-03-29T14:08:32.394 INFO:tasks.rados.rados.0.smithi017.stdout:1476: finishing write tid 2 to smithi017107884-17 2023-03-29T14:08:32.394 INFO:tasks.rados.rados.0.smithi017.stdout:1476: finishing write tid 3 to smithi017107884-17 2023-03-29T14:08:32.395 INFO:tasks.rados.rados.0.smithi017.stdout:1476: finishing write tid 4 to smithi017107884-17 2023-03-29T14:08:32.396 INFO:tasks.rados.rados.0.smithi017.stdout:1476: finishing write tid 5 to smithi017107884-17 2023-03-29T14:08:32.396 INFO:tasks.rados.rados.0.smithi017.stdout:1476: oid 17 updating version 0 to 237 2023-03-29T14:08:32.396 INFO:tasks.rados.rados.0.smithi017.stdout:1476: oid 17 version 237 is already newer than 237 2023-03-29T14:08:32.396 INFO:tasks.rados.rados.0.smithi017.stdout:1476: oid 17 version 237 is already newer than 237 2023-03-29T14:08:32.396 INFO:tasks.rados.rados.0.smithi017.stdout:1476: oid 17 version 237 is already newer than 237 2023-03-29T14:08:32.396 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 237 (ObjNum 709 snap 174 seq_num 709) dirty exists 2023-03-29T14:08:32.396 INFO:tasks.rados.rados.0.smithi017.stdout:1476: left oid 17 (ObjNum 709 snap 174 seq_num 709) 2023-03-29T14:08:32.548 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.3,3.6] on osd.0 to force-recovery; instructing pg(s) [1.0] on osd.1 to force-recovery; instructing pg(s) [3.2,3.f] on osd.3 to force-recovery; 2023-03-29T14:08:32.548 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:08:33.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:33.250 INFO:tasks.rados.rados.0.smithi017.stdout:1476: done (1 left) 2023-03-29T14:08:33.250 INFO:tasks.rados.rados.0.smithi017.stdout:1477: done (0 left) 2023-03-29T14:08:33.250 INFO:tasks.rados.rados.0.smithi017.stdout:1478: write (excl) oid 3 current snap is 175 2023-03-29T14:08:33.250 INFO:tasks.rados.rados.0.smithi017.stdout:1478: seq_num 710 ranges {732970=647619,2097627=611905,3231266=437835} 2023-03-29T14:08:33.257 INFO:tasks.rados.rados.0.smithi017.stdout:1478: writing smithi017107884-3 from 732970 to 1380589 tid 1 2023-03-29T14:08:33.264 INFO:tasks.rados.rados.0.smithi017.stdout:1478: writing smithi017107884-3 from 2097627 to 2709532 tid 2 2023-03-29T14:08:33.270 INFO:tasks.rados.rados.0.smithi017.stdout:1478: writing smithi017107884-3 from 3231266 to 3669101 tid 3 2023-03-29T14:08:33.270 INFO:tasks.rados.rados.0.smithi017.stdout:1479: write (excl) oid 12 current snap is 175 2023-03-29T14:08:33.270 INFO:tasks.rados.rados.0.smithi017.stdout:1479: seq_num 711 ranges {749777=722868,2156109=471916,3258689=649107} 2023-03-29T14:08:33.278 INFO:tasks.rados.rados.0.smithi017.stdout:1479: writing smithi017107884-12 from 749777 to 1472645 tid 1 2023-03-29T14:08:33.285 INFO:tasks.rados.rados.0.smithi017.stdout:1479: writing smithi017107884-12 from 2156109 to 2628025 tid 2 2023-03-29T14:08:33.293 INFO:tasks.rados.rados.0.smithi017.stdout:1479: writing smithi017107884-12 from 3258689 to 3907796 tid 3 2023-03-29T14:08:33.293 INFO:tasks.rados.rados.0.smithi017.stdout:1480: write oid 21 current snap is 175 2023-03-29T14:08:33.293 INFO:tasks.rados.rados.0.smithi017.stdout:1480: seq_num 712 ranges {766584=798117,2098238=615574,3169759=460379} 2023-03-29T14:08:33.301 INFO:tasks.rados.rados.0.smithi017.stdout:1480: writing smithi017107884-21 from 766584 to 1564701 tid 1 2023-03-29T14:08:33.308 INFO:tasks.rados.rados.0.smithi017.stdout:1480: writing smithi017107884-21 from 2098238 to 2713812 tid 2 2023-03-29T14:08:33.313 INFO:tasks.rados.rados.0.smithi017.stdout:1480: writing smithi017107884-21 from 3169759 to 3630138 tid 3 2023-03-29T14:08:33.313 INFO:tasks.rados.rados.0.smithi017.stdout:1481: write oid 16 current snap is 175 2023-03-29T14:08:33.313 INFO:tasks.rados.rados.0.smithi017.stdout:1481: seq_num 713 ranges {783391=473366,1756720=475585,2797182=788004} 2023-03-29T14:08:33.320 INFO:tasks.rados.rados.0.smithi017.stdout:1481: writing smithi017107884-16 from 783391 to 1256757 tid 1 2023-03-29T14:08:33.325 INFO:tasks.rados.rados.0.smithi017.stdout:1481: writing smithi017107884-16 from 1756720 to 2232305 tid 2 2023-03-29T14:08:33.333 INFO:tasks.rados.rados.0.smithi017.stdout:1481: writing smithi017107884-16 from 2797182 to 3585186 tid 3 2023-03-29T14:08:33.333 INFO:tasks.rados.rados.0.smithi017.stdout:1482: delete oid 18 current snap is 175 2023-03-29T14:08:33.357 INFO:tasks.rados.rados.0.smithi017.stdout:1482: done (4 left) 2023-03-29T14:08:33.357 INFO:tasks.rados.rados.0.smithi017.stdout:1483: snap_create 2023-03-29T14:08:33.376 INFO:tasks.rados.rados.0.smithi017.stdout:1481: finishing write tid 1 to smithi017107884-16 2023-03-29T14:08:33.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:33.387+0000 7ff81073a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:33.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:33.392+0000 7ff81073a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:33.402 INFO:tasks.rados.rados.0.smithi017.stdout:1478: finishing write tid 1 to smithi017107884-3 2023-03-29T14:08:33.409 INFO:tasks.rados.rados.0.smithi017.stdout:1481: finishing write tid 2 to smithi017107884-16 2023-03-29T14:08:33.410 INFO:tasks.rados.rados.0.smithi017.stdout:1481: finishing write tid 3 to smithi017107884-16 2023-03-29T14:08:33.410 INFO:tasks.rados.rados.0.smithi017.stdout:1481: finishing write tid 4 to smithi017107884-16 2023-03-29T14:08:33.412 INFO:tasks.rados.rados.0.smithi017.stdout:1481: finishing write tid 5 to smithi017107884-16 2023-03-29T14:08:33.412 INFO:tasks.rados.rados.0.smithi017.stdout:1481: oid 16 updating version 0 to 112 2023-03-29T14:08:33.412 INFO:tasks.rados.rados.0.smithi017.stdout:1481: oid 16 version 112 is already newer than 112 2023-03-29T14:08:33.412 INFO:tasks.rados.rados.0.smithi017.stdout:1481: oid 16 version 112 is already newer than 112 2023-03-29T14:08:33.412 INFO:tasks.rados.rados.0.smithi017.stdout:1481: oid 16 version 112 is already newer than 112 2023-03-29T14:08:33.412 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 112 (ObjNum 713 snap 175 seq_num 713) dirty exists 2023-03-29T14:08:33.412 INFO:tasks.rados.rados.0.smithi017.stdout:1481: left oid 16 (ObjNum 713 snap 175 seq_num 713) 2023-03-29T14:08:33.439 INFO:tasks.rados.rados.0.smithi017.stdout:1478: finishing write tid 2 to smithi017107884-3 2023-03-29T14:08:33.449 INFO:tasks.rados.rados.0.smithi017.stdout:1478: finishing write tid 3 to smithi017107884-3 2023-03-29T14:08:33.457 INFO:tasks.rados.rados.0.smithi017.stdout:1480: finishing write tid 1 to smithi017107884-21 2023-03-29T14:08:33.465 INFO:tasks.rados.rados.0.smithi017.stdout:1478: finishing write tid 4 to smithi017107884-3 2023-03-29T14:08:33.473 INFO:tasks.rados.rados.0.smithi017.stdout:1478: finishing write tid 5 to smithi017107884-3 2023-03-29T14:08:33.473 INFO:tasks.rados.rados.0.smithi017.stdout:1478: oid 3 updating version 0 to 309 2023-03-29T14:08:33.473 INFO:tasks.rados.rados.0.smithi017.stdout:1478: oid 3 version 309 is already newer than 309 2023-03-29T14:08:33.473 INFO:tasks.rados.rados.0.smithi017.stdout:1478: oid 3 version 309 is already newer than 309 2023-03-29T14:08:33.473 INFO:tasks.rados.rados.0.smithi017.stdout:1478: oid 3 version 309 is already newer than 305 2023-03-29T14:08:33.473 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 309 (ObjNum 710 snap 175 seq_num 710) dirty exists 2023-03-29T14:08:33.473 INFO:tasks.rados.rados.0.smithi017.stdout:1478: left oid 3 (ObjNum 710 snap 175 seq_num 710) 2023-03-29T14:08:33.474 INFO:tasks.rados.rados.0.smithi017.stdout:1480: finishing write tid 2 to smithi017107884-21 2023-03-29T14:08:33.516 INFO:tasks.rados.rados.0.smithi017.stdout:1479: finishing write tid 1 to smithi017107884-12 2023-03-29T14:08:33.520 INFO:tasks.rados.rados.0.smithi017.stdout:1480: finishing write tid 3 to smithi017107884-21 2023-03-29T14:08:33.526 INFO:tasks.rados.rados.0.smithi017.stdout:1480: finishing write tid 4 to smithi017107884-21 2023-03-29T14:08:33.527 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:33.528 INFO:tasks.rados.rados.0.smithi017.stdout:1480: finishing write tid 5 to smithi017107884-21 2023-03-29T14:08:33.528 INFO:tasks.rados.rados.0.smithi017.stdout:1480: oid 21 updating version 0 to 172 2023-03-29T14:08:33.528 INFO:tasks.rados.rados.0.smithi017.stdout:1480: oid 21 version 172 is already newer than 172 2023-03-29T14:08:33.528 INFO:tasks.rados.rados.0.smithi017.stdout:1480: oid 21 version 172 is already newer than 172 2023-03-29T14:08:33.528 INFO:tasks.rados.rados.0.smithi017.stdout:1480: oid 21 version 172 is already newer than 172 2023-03-29T14:08:33.528 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 172 (ObjNum 712 snap 175 seq_num 712) dirty exists 2023-03-29T14:08:33.528 INFO:tasks.rados.rados.0.smithi017.stdout:1480: left oid 21 (ObjNum 712 snap 175 seq_num 712) 2023-03-29T14:08:33.531 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:33.531 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:33.543 INFO:tasks.rados.rados.0.smithi017.stdout:1479: finishing write tid 2 to smithi017107884-12 2023-03-29T14:08:33.582 INFO:tasks.rados.rados.0.smithi017.stdout:1479: finishing write tid 3 to smithi017107884-12 2023-03-29T14:08:33.584 INFO:tasks.rados.rados.0.smithi017.stdout:1479: finishing write tid 4 to smithi017107884-12 2023-03-29T14:08:33.586 INFO:tasks.rados.rados.0.smithi017.stdout:1479: finishing write tid 5 to smithi017107884-12 2023-03-29T14:08:33.586 INFO:tasks.rados.rados.0.smithi017.stdout:1479: oid 12 updating version 0 to 206 2023-03-29T14:08:33.586 INFO:tasks.rados.rados.0.smithi017.stdout:1479: oid 12 version 206 is already newer than 203 2023-03-29T14:08:33.586 INFO:tasks.rados.rados.0.smithi017.stdout:1479: oid 12 version 206 is already newer than 206 2023-03-29T14:08:33.586 INFO:tasks.rados.rados.0.smithi017.stdout:1479: oid 12 version 206 is already newer than 206 2023-03-29T14:08:33.586 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 206 (ObjNum 711 snap 175 seq_num 711) dirty exists 2023-03-29T14:08:33.586 INFO:tasks.rados.rados.0.smithi017.stdout:1479: left oid 12 (ObjNum 711 snap 175 seq_num 711) 2023-03-29T14:08:33.717 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~204s 2023-03-29T14:08:34.285 INFO:tasks.rados.rados.0.smithi017.stdout:1478: done (4 left) 2023-03-29T14:08:34.286 INFO:tasks.rados.rados.0.smithi017.stdout:1479: done (3 left) 2023-03-29T14:08:34.286 INFO:tasks.rados.rados.0.smithi017.stdout:1480: done (2 left) 2023-03-29T14:08:34.286 INFO:tasks.rados.rados.0.smithi017.stdout:1481: done (1 left) 2023-03-29T14:08:34.286 INFO:tasks.rados.rados.0.smithi017.stdout:1483: done (0 left) 2023-03-29T14:08:34.286 INFO:tasks.rados.rados.0.smithi017.stdout:1484: write (excl) oid 45 current snap is 176 2023-03-29T14:08:34.286 INFO:tasks.rados.rados.0.smithi017.stdout:1484: seq_num 715 ranges {417005=740217,1590037=426968} 2023-03-29T14:08:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:1484: writing smithi017107884-45 from 417005 to 1157222 tid 1 2023-03-29T14:08:34.296 INFO:tasks.rados.rados.0.smithi017.stdout:1484: writing smithi017107884-45 from 1590037 to 2017005 tid 2 2023-03-29T14:08:34.297 INFO:tasks.rados.rados.0.smithi017.stdout:1485: snap_remove snap 168 2023-03-29T14:08:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1485: done (1 left) 2023-03-29T14:08:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1486: write oid 31 current snap is 176 2023-03-29T14:08:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1484: finishing write tid 1 to smithi017107884-45 2023-03-29T14:08:35.289 INFO:tasks.rados.rados.0.smithi017.stdout:1486: seq_num 716 ranges {433812=415466,1648519=385293} 2023-03-29T14:08:35.294 INFO:tasks.rados.rados.0.smithi017.stdout:1486: writing smithi017107884-31 from 433812 to 849278 tid 1 2023-03-29T14:08:35.301 INFO:tasks.rados.rados.0.smithi017.stdout:1486: writing smithi017107884-31 from 1648519 to 2033812 tid 2 2023-03-29T14:08:35.301 INFO:tasks.rados.rados.0.smithi017.stdout:1487: read oid 40 snap -1 2023-03-29T14:08:35.301 INFO:tasks.rados.rados.0.smithi017.stdout:1487: expect (ObjNum 685 snap 166 seq_num 685) 2023-03-29T14:08:35.301 INFO:tasks.rados.rados.0.smithi017.stdout:1488: read oid 24 snap -1 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1488: expect deleted 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1489: delete oid 9 current snap is 176 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1484: finishing write tid 2 to smithi017107884-45 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1484: finishing write tid 3 to smithi017107884-45 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1484: finishing write tid 4 to smithi017107884-45 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1484: oid 45 updating version 0 to 199 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1484: oid 45 updating version 199 to 201 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1484: oid 45 version 201 is already newer than 201 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 201 (ObjNum 715 snap 176 seq_num 715) dirty exists 2023-03-29T14:08:35.302 INFO:tasks.rados.rados.0.smithi017.stdout:1484: left oid 45 (ObjNum 715 snap 176 seq_num 715) 2023-03-29T14:08:35.355 INFO:tasks.rados.rados.0.smithi017.stdout:1487: expect (ObjNum 685 snap 166 seq_num 685) 2023-03-29T14:08:35.532 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:35.547 INFO:tasks.rados.rados.0.smithi017.stdout:1486: finishing write tid 1 to smithi017107884-31 2023-03-29T14:08:35.547 INFO:tasks.rados.rados.0.smithi017.stdout:1486: finishing write tid 2 to smithi017107884-31 2023-03-29T14:08:35.547 INFO:tasks.rados.rados.0.smithi017.stdout:1486: finishing write tid 3 to smithi017107884-31 2023-03-29T14:08:35.547 INFO:tasks.rados.rados.0.smithi017.stdout:1486: finishing write tid 4 to smithi017107884-31 2023-03-29T14:08:35.547 INFO:tasks.rados.rados.0.smithi017.stdout:1486: oid 31 updating version 0 to 124 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1486: oid 31 version 124 is already newer than 124 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1486: oid 31 version 124 is already newer than 124 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 124 (ObjNum 716 snap 176 seq_num 716) dirty exists 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1486: left oid 31 (ObjNum 716 snap 176 seq_num 716) 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1484: done (4 left) 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1486: done (3 left) 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1487: done (2 left) 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1488: done (1 left) 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1489: done (0 left) 2023-03-29T14:08:35.548 INFO:tasks.rados.rados.0.smithi017.stdout:1490: delete oid 36 current snap is 176 2023-03-29T14:08:35.601 INFO:tasks.rados.rados.0.smithi017.stdout:1490: done (0 left) 2023-03-29T14:08:35.601 INFO:tasks.rados.rados.0.smithi017.stdout:1491: snap_remove snap 173 2023-03-29T14:08:35.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:35.663+0000 7fe25c1e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:35.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:35.668+0000 7fe25c1e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:35.730 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:35.733 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:35.733 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:36.293 INFO:tasks.rados.rados.0.smithi017.stdout:1491: done (0 left) 2023-03-29T14:08:36.293 INFO:tasks.rados.rados.0.smithi017.stdout:1492: snap_create 2023-03-29T14:08:37.293 INFO:tasks.rados.rados.0.smithi017.stdout:1492: done (0 left) 2023-03-29T14:08:37.294 INFO:tasks.rados.rados.0.smithi017.stdout:1493: write (excl) oid 23 current snap is 177 2023-03-29T14:08:37.294 INFO:tasks.rados.rados.0.smithi017.stdout:1493: seq_num 719 ranges {484233=641213,1707612=376621} 2023-03-29T14:08:37.300 INFO:tasks.rados.rados.0.smithi017.stdout:1493: writing smithi017107884-23 from 484233 to 1125446 tid 1 2023-03-29T14:08:37.305 INFO:tasks.rados.rados.0.smithi017.stdout:1493: writing smithi017107884-23 from 1707612 to 2084233 tid 2 2023-03-29T14:08:37.305 INFO:tasks.rados.rados.0.smithi017.stdout:1494: read oid 33 snap 171 2023-03-29T14:08:37.305 INFO:tasks.rados.rados.0.smithi017.stdout:1494: expect (ObjNum 663 snap 162 seq_num 663) 2023-03-29T14:08:37.306 INFO:tasks.rados.rados.0.smithi017.stdout:1495: write oid 12 current snap is 177 2023-03-29T14:08:37.306 INFO:tasks.rados.rados.0.smithi017.stdout:1495: seq_num 720 ranges {501040=716462,1766094=334946} 2023-03-29T14:08:37.312 INFO:tasks.rados.rados.0.smithi017.stdout:1495: writing smithi017107884-12 from 501040 to 1217502 tid 1 2023-03-29T14:08:37.318 INFO:tasks.rados.rados.0.smithi017.stdout:1495: writing smithi017107884-12 from 1766094 to 2101040 tid 2 2023-03-29T14:08:37.318 INFO:tasks.rados.rados.0.smithi017.stdout:1496: read oid 44 snap -1 2023-03-29T14:08:37.318 INFO:tasks.rados.rados.0.smithi017.stdout:1496: expect (ObjNum 698 snap 168 seq_num 698) 2023-03-29T14:08:37.318 INFO:tasks.rados.rados.0.smithi017.stdout:1497: write (excl) oid 5 current snap is 177 2023-03-29T14:08:37.318 INFO:tasks.rados.rados.0.smithi017.stdout:1497: seq_num 721 ranges {517847=791711,2108223=9624} 2023-03-29T14:08:37.325 INFO:tasks.rados.rados.0.smithi017.stdout:1497: writing smithi017107884-5 from 517847 to 1309558 tid 1 2023-03-29T14:08:37.329 INFO:tasks.rados.rados.0.smithi017.stdout:1497: writing smithi017107884-5 from 2108223 to 2117847 tid 2 2023-03-29T14:08:37.329 INFO:tasks.rados.rados.0.smithi017.stdout:1498: snap_remove snap 166 2023-03-29T14:08:37.560 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:37.560 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:37.560 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:37.691+0000 7fe6b8e4a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:37.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:37.698+0000 7fe6b8e4a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:37.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:37.865 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:37.863+0000 7fe8ff209700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:37.868 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:37.868+0000 7fe8ff209700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:37.961 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:38.092 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:38.092 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:38.111 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:38.114 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:38.131 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":559,"stamp":"2023-03-29T14:08:37.292410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":254303792,"num_objects":812,"num_object_clones":716,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":3321,"num_read_kb":1920954,"num_write":4608,"num_write_kb":1268498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2429,"ondisk_log_size":2429,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1608220,"kb_used_data":1528764,"kb_used_omap":0,"kb_used_meta":79424,"kb_avail":375879140,"statfs":{"total":386547056640,"available":384900239360,"internally_reserved":0,"allocated":1565454336,"data_stored":1914730303,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":81330176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12135314,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-4,"num_read":22,"num_read_kb":5326,"num_write":66,"num_write_kb":21084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.044638"},"pg_stats":[{"pgid":"3.f","version":"366'112","reported_seq":338,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:35.316724+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:35.316724+0000","last_peered":"2023-03-29T14:08:35.316724+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:35.316724+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15664652,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":65942,"num_write":138,"num_write_kb":40018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":20}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"361'44","reported_seq":378,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:34.339678+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:34.339678+0000","last_peered":"2023-03-29T14:08:34.339678+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:34.339678+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7563414,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":25432,"num_write":82,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"365'237","reported_seq":836,"reported_epoch":369,"state":"active+clean","last_fresh":"2023-03-29T14:08:35.315334+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:35.315334+0000","last_peered":"2023-03-29T14:08:35.315334+0000","last_clean":"2023-03-29T14:08:35.315334+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:35.315334+0000","last_undegraded":"2023-03-29T14:08:35.315334+0000","last_fullsized":"2023-03-29T14:08:35.315334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32976654,"num_objects":84,"num_object_clones":74,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":341,"num_read_kb":239436,"num_write":458,"num_write_kb":136364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"368'225","reported_seq":810,"reported_epoch":369,"state":"active+clean","last_fresh":"2023-03-29T14:08:35.355982+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:35.355982+0000","last_peered":"2023-03-29T14:08:35.355982+0000","last_clean":"2023-03-29T14:08:35.355982+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:35.355982+0000","last_undegraded":"2023-03-29T14:08:35.355982+0000","last_fullsized":"2023-03-29T14:08:35.355982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14824302,"num_objects":90,"num_object_clones":82,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":331,"num_read_kb":206398,"num_write":436,"num_write_kb":125006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"367'201","reported_seq":724,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-03-29T14:08:34.485666+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:34.485666+0000","last_peered":"2023-03-29T14:08:34.485666+0000","last_clean":"2023-03-29T14:08:34.485666+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:34.485666+0000","last_undegraded":"2023-03-29T14:08:34.485666+0000","last_fullsized":"2023-03-29T14:08:34.485666+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412910,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":253,"num_read_kb":145261,"num_write":400,"num_write_kb":117558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":309,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-03-29T14:08:35.306418+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:35.306418+0000","last_peered":"2023-03-29T14:08:35.306418+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:35.306418+0000","last_undegraded":"2023-03-29T14:08:35.306418+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"368'124","reported_seq":568,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:35.404924+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:35.404924+0000","last_peered":"2023-03-29T14:08:35.404924+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:35.404924+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067624,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":146,"num_read_kb":67876,"num_write":222,"num_write_kb":63420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":44}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":483,"reported_epoch":369,"state":"active+clean","last_fresh":"2023-03-29T14:08:35.320188+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:35.320188+0000","last_peered":"2023-03-29T14:08:35.320188+0000","last_clean":"2023-03-29T14:08:35.320188+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:35.320188+0000","last_undegraded":"2023-03-29T14:08:35.320188+0000","last_fullsized":"2023-03-29T14:08:35.320188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":289,"reported_epoch":368,"state":"active+undersized","last_fresh":"2023-03-29T14:08:34.357224+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:34.357224+0000","last_peered":"2023-03-29T14:08:34.357224+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:34.357224+0000","last_undegraded":"2023-03-29T14:08:34.357224+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"334'86","reported_seq":253,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:34.535517+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:34.535517+0000","last_peered":"2023-03-29T14:08:34.535517+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:34.535517+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050784,"num_objects":16,"num_object_clones":16,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":37,"num_read_kb":12593,"num_write":96,"num_write_kb":25750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":312,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-03-29T14:08:35.314611+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:35.314611+0000","last_peered":"2023-03-29T14:08:35.314611+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:35.314611+0000","last_undegraded":"2023-03-29T14:08:35.314611+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"361'93","reported_seq":508,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:34.533505+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:34.533505+0000","last_peered":"2023-03-29T14:08:34.533505+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:34.533505+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731484,"num_objects":38,"num_object_clones":34,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":149,"num_read_kb":100906,"num_write":182,"num_write_kb":52894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":368,"state":"active+undersized","last_fresh":"2023-03-29T14:08:34.316594+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:34.316594+0000","last_peered":"2023-03-29T14:08:34.316594+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:34.316594+0000","last_undegraded":"2023-03-29T14:08:34.316594+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"361'129","reported_seq":583,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:35.553291+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:35.553291+0000","last_peered":"2023-03-29T14:08:35.553291+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:35.553291+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10925078,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":195,"num_read_kb":112152,"num_write":252,"num_write_kb":74172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":312,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-03-29T14:08:35.321057+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:35.321057+0000","last_peered":"2023-03-29T14:08:35.321057+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:35.321057+0000","last_undegraded":"2023-03-29T14:08:35.321057+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":514,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:34.334567+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:34.334567+0000","last_peered":"2023-03-29T14:08:34.334567+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:34.334567+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"366'172","reported_seq":737,"reported_epoch":369,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:35.311119+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:35.311119+0000","last_peered":"2023-03-29T14:08:35.311119+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:35.311119+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22118804,"num_objects":60,"num_object_clones":52,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":321,"num_read_kb":215756,"num_write":334,"num_write_kb":97678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":60}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":312,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-03-29T14:08:35.315916+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:35.315916+0000","last_peered":"2023-03-29T14:08:35.315916+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:35.315916+0000","last_undegraded":"2023-03-29T14:08:35.315916+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"366'309","reported_seq":991,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:34.522433+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:34.522433+0000","last_peered":"2023-03-29T14:08:34.522433+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:34.522433+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33783390,"num_objects":108,"num_object_clones":94,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":466,"num_read_kb":277057,"num_write":580,"num_write_kb":168654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":108}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":289,"reported_epoch":368,"state":"active+undersized","last_fresh":"2023-03-29T14:08:34.319614+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:34.319614+0000","last_peered":"2023-03-29T14:08:34.319614+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:34.319614+0000","last_undegraded":"2023-03-29T14:08:34.319614+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"358'114","reported_seq":530,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-03-29T14:08:34.351686+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:34.351686+0000","last_peered":"2023-03-29T14:08:34.351686+0000","last_clean":"2023-03-29T14:08:34.351686+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:34.351686+0000","last_undegraded":"2023-03-29T14:08:34.351686+0000","last_fullsized":"2023-03-29T14:08:34.351686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286478,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":70184,"num_write":234,"num_write_kb":68058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"366'206","reported_seq":729,"reported_epoch":368,"state":"active+clean","last_fresh":"2023-03-29T14:08:34.346270+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:34.346270+0000","last_peered":"2023-03-29T14:08:34.346270+0000","last_clean":"2023-03-29T14:08:34.346270+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:34.346270+0000","last_undegraded":"2023-03-29T14:08:34.346270+0000","last_fullsized":"2023-03-29T14:08:34.346270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29043486,"num_objects":70,"num_object_clones":60,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":264,"num_read_kb":159206,"num_write":380,"num_write_kb":111830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":289,"reported_epoch":368,"state":"active+undersized","last_fresh":"2023-03-29T14:08:34.347382+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:34.347382+0000","last_peered":"2023-03-29T14:08:34.347382+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:34.347382+0000","last_undegraded":"2023-03-29T14:08:34.347382+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":312,"reported_epoch":369,"state":"active+undersized","last_fresh":"2023-03-29T14:08:35.319280+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:35.319280+0000","last_peered":"2023-03-29T14:08:35.319280+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:35.319280+0000","last_undegraded":"2023-03-29T14:08:35.319280+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"352'201","reported_seq":738,"reported_epoch":368,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:34.515620+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:34.515620+0000","last_peered":"2023-03-29T14:08:34.515620+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:34.515620+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28942578,"num_objects":70,"num_object_clones":62,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":164668,"num_write":388,"num_write_kb":114584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":253123056,"num_objects":808,"num_object_clones":716,"num_object_copies":2424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":3179,"num_read_kb":1920834,"num_write":4334,"num_write_kb":1264710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2332,"ondisk_log_size":2332,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":465040,"kb_used_data":443144,"kb_used_omap":0,"kb_used_meta":21888,"kb_avail":93906800,"statfs":{"total":96636764160,"available":96160563200,"internally_reserved":0,"allocated":453779456,"data_stored":556337135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22413312},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706053,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":519108,"kb_used_data":495932,"kb_used_omap":0,"kb_used_meta":23168,"kb_avail":93852732,"statfs":{"total":96636764160,"available":96105197568,"internally_reserved":0,"allocated":507834368,"data_stored":625053626,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23724032},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706053,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":468128,"kb_used_data":445848,"kb_used_omap":0,"kb_used_meta":22272,"kb_avail":93903712,"statfs":{"total":96636764160,"available":96157401088,"internally_reserved":0,"allocated":456548352,"data_stored":560107317,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":22806528},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:38.135 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.5 3.3 3.0 1.0 3.1 2023-03-29T14:08:38.303 INFO:tasks.rados.rados.0.smithi017.stdout:1498: done (5 left) 2023-03-29T14:08:38.303 INFO:tasks.rados.rados.0.smithi017.stdout:1499: delete oid 1 current snap is 177 2023-03-29T14:08:38.304 INFO:tasks.rados.rados.0.smithi017.stdout:1493: finishing write tid 1 to smithi017107884-23 2023-03-29T14:08:38.304 INFO:tasks.rados.rados.0.smithi017.stdout:1493: finishing write tid 2 to smithi017107884-23 2023-03-29T14:08:38.304 INFO:tasks.rados.rados.0.smithi017.stdout:1494: expect (ObjNum 663 snap 162 seq_num 663) 2023-03-29T14:08:38.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:38.300+0000 7f0dee18a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:38.313 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:38.313+0000 7f0dee18a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1493: finishing write tid 3 to smithi017107884-23 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1493: finishing write tid 4 to smithi017107884-23 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1493: oid 23 updating version 0 to 312 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1493: oid 23 version 312 is already newer than 310 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1493: oid 23 version 312 is already newer than 312 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 312 (ObjNum 719 snap 177 seq_num 719) dirty exists 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1493: left oid 23 (ObjNum 719 snap 177 seq_num 719) 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1493: done (5 left) 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1494: done (4 left) 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1499: done (3 left) 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1500: read oid 24 snap -1 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1500: expect deleted 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1501: write (excl) oid 25 current snap is 177 2023-03-29T14:08:38.432 INFO:tasks.rados.rados.0.smithi017.stdout:1501: seq_num 723 ranges {551461=658562,1941540=209921} 2023-03-29T14:08:38.442 INFO:tasks.rados.rados.0.smithi017.stdout:1501: writing smithi017107884-25 from 551461 to 1210023 tid 1 2023-03-29T14:08:38.449 INFO:tasks.rados.rados.0.smithi017.stdout:1501: writing smithi017107884-25 from 1941540 to 2151461 tid 2 2023-03-29T14:08:38.449 INFO:tasks.rados.rados.0.smithi017.stdout:1502: delete oid 41 current snap is 177 2023-03-29T14:08:38.449 INFO:tasks.rados.rados.0.smithi017.stdout:1496: expect (ObjNum 698 snap 168 seq_num 698) 2023-03-29T14:08:38.528 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3,3.5] on osd.0 to force-backfill; instructing pg(s) [1.0,3.0] on osd.1 to force-backfill; 2023-03-29T14:08:38.528 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:08:38.685 INFO:tasks.rados.rados.0.smithi017.stdout:1495: finishing write tid 1 to smithi017107884-12 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: finishing write tid 2 to smithi017107884-12 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: finishing write tid 3 to smithi017107884-12 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: finishing write tid 4 to smithi017107884-12 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: oid 12 updating version 0 to 210 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: oid 12 version 210 is already newer than 210 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: oid 12 version 210 is already newer than 210 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 210 (ObjNum 720 snap 177 seq_num 720) dirty exists 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: left oid 12 (ObjNum 720 snap 177 seq_num 720) 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1497: finishing write tid 1 to smithi017107884-5 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1497: finishing write tid 2 to smithi017107884-5 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1497: finishing write tid 3 to smithi017107884-5 2023-03-29T14:08:38.686 INFO:tasks.rados.rados.0.smithi017.stdout:1495: done (5 left) 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1496: done (4 left) 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1502: done (3 left) 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1503: read oid 11 snap -1 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1503: expect (ObjNum 674 snap 164 seq_num 674) 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1504: read oid 37 snap 174 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1504: expect deleted 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1505: write oid 9 current snap is 177 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1497: finishing write tid 4 to smithi017107884-5 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1497: oid 5 updating version 0 to 214 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1497: oid 5 version 214 is already newer than 214 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1497: oid 5 version 214 is already newer than 211 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 214 (ObjNum 721 snap 177 seq_num 721) dirty exists 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1497: left oid 5 (ObjNum 721 snap 177 seq_num 721) 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1501: finishing write tid 1 to smithi017107884-25 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1501: finishing write tid 2 to smithi017107884-25 2023-03-29T14:08:38.687 INFO:tasks.rados.rados.0.smithi017.stdout:1501: finishing write tid 3 to smithi017107884-25 2023-03-29T14:08:38.688 INFO:tasks.rados.rados.0.smithi017.stdout:1501: finishing write tid 4 to smithi017107884-25 2023-03-29T14:08:38.688 INFO:tasks.rados.rados.0.smithi017.stdout:1501: oid 25 updating version 0 to 228 2023-03-29T14:08:38.688 INFO:tasks.rados.rados.0.smithi017.stdout:1501: oid 25 version 228 is already newer than 228 2023-03-29T14:08:38.688 INFO:tasks.rados.rados.0.smithi017.stdout:1501: oid 25 version 228 is already newer than 226 2023-03-29T14:08:38.688 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 228 (ObjNum 723 snap 177 seq_num 723) dirty exists 2023-03-29T14:08:38.688 INFO:tasks.rados.rados.0.smithi017.stdout:1501: left oid 25 (ObjNum 723 snap 177 seq_num 723) 2023-03-29T14:08:38.688 INFO:tasks.rados.rados.0.smithi017.stdout:1505: seq_num 725 ranges {585075=409060,1542151=642924} 2023-03-29T14:08:38.690 INFO:tasks.rados.rados.0.smithi017.stdout:1505: writing smithi017107884-9 from 585075 to 994135 tid 1 2023-03-29T14:08:38.697 INFO:tasks.rados.rados.0.smithi017.stdout:1505: writing smithi017107884-9 from 1542151 to 2185075 tid 2 2023-03-29T14:08:38.697 INFO:tasks.rados.rados.0.smithi017.stdout:1497: done (5 left) 2023-03-29T14:08:38.697 INFO:tasks.rados.rados.0.smithi017.stdout:1500: done (4 left) 2023-03-29T14:08:38.698 INFO:tasks.rados.rados.0.smithi017.stdout:1501: done (3 left) 2023-03-29T14:08:38.698 INFO:tasks.rados.rados.0.smithi017.stdout:1506: write oid 6 current snap is 177 2023-03-29T14:08:38.698 INFO:tasks.rados.rados.0.smithi017.stdout:1506: seq_num 726 ranges {601882=484309,1600633=524904} 2023-03-29T14:08:38.705 INFO:tasks.rados.rados.0.smithi017.stdout:1506: writing smithi017107884-6 from 601882 to 1086191 tid 1 2023-03-29T14:08:38.711 INFO:tasks.rados.rados.0.smithi017.stdout:1506: writing smithi017107884-6 from 1600633 to 2125537 tid 2 2023-03-29T14:08:38.711 INFO:tasks.rados.rados.0.smithi017.stdout:1507: read oid 27 snap 169 2023-03-29T14:08:38.711 INFO:tasks.rados.rados.0.smithi017.stdout:1507: expect (ObjNum 694 snap 167 seq_num 694) 2023-03-29T14:08:38.711 INFO:tasks.rados.rados.0.smithi017.stdout:1508: write (excl) oid 34 current snap is 177 2023-03-29T14:08:38.711 INFO:tasks.rados.rados.0.smithi017.stdout:1508: seq_num 727 ranges {618689=559558,1659115=559574} 2023-03-29T14:08:38.718 INFO:tasks.rados.rados.0.smithi017.stdout:1508: writing smithi017107884-34 from 618689 to 1178247 tid 1 2023-03-29T14:08:38.724 INFO:tasks.rados.rados.0.smithi017.stdout:1508: writing smithi017107884-34 from 1659115 to 2218689 tid 2 2023-03-29T14:08:38.724 INFO:tasks.rados.rados.0.smithi017.stdout:1509: snap_remove snap 164 2023-03-29T14:08:39.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~209s 2023-03-29T14:08:39.303 INFO:tasks.rados.rados.0.smithi017.stdout:1509: done (6 left) 2023-03-29T14:08:39.303 INFO:tasks.rados.rados.0.smithi017.stdout:1510: write oid 49 current snap is 177 2023-03-29T14:08:39.303 INFO:tasks.rados.rados.0.smithi017.stdout:1510: seq_num 728 ranges {635496=634807,1717597=517899} 2023-03-29T14:08:39.310 INFO:tasks.rados.rados.0.smithi017.stdout:1510: writing smithi017107884-49 from 635496 to 1270303 tid 1 2023-03-29T14:08:39.316 INFO:tasks.rados.rados.0.smithi017.stdout:1510: writing smithi017107884-49 from 1717597 to 2235496 tid 2 2023-03-29T14:08:39.316 INFO:tasks.rados.rados.0.smithi017.stdout:1511: read oid 20 snap -1 2023-03-29T14:08:39.316 INFO:tasks.rados.rados.0.smithi017.stdout:1511: expect (ObjNum 699 snap 168 seq_num 699) 2023-03-29T14:08:39.316 INFO:tasks.rados.rados.0.smithi017.stdout:1512: write oid 30 current snap is 177 2023-03-29T14:08:39.316 INFO:tasks.rados.rados.0.smithi017.stdout:1512: seq_num 729 ranges {652303=710056,2059726=192577} 2023-03-29T14:08:39.323 INFO:tasks.rados.rados.0.smithi017.stdout:1512: writing smithi017107884-30 from 652303 to 1362359 tid 1 2023-03-29T14:08:39.328 INFO:tasks.rados.rados.0.smithi017.stdout:1512: writing smithi017107884-30 from 2059726 to 2252303 tid 2 2023-03-29T14:08:39.328 INFO:tasks.rados.rados.0.smithi017.stdout:1513: delete oid 44 current snap is 177 2023-03-29T14:08:39.328 INFO:tasks.rados.rados.0.smithi017.stdout:1503: expect (ObjNum 674 snap 164 seq_num 674) 2023-03-29T14:08:39.466 INFO:tasks.rados.rados.0.smithi017.stdout:1505: finishing write tid 1 to smithi017107884-9 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1506: finishing write tid 1 to smithi017107884-6 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1505: finishing write tid 2 to smithi017107884-9 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1505: finishing write tid 3 to smithi017107884-9 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1505: finishing write tid 4 to smithi017107884-9 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1505: oid 9 updating version 0 to 231 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1505: oid 9 version 231 is already newer than 231 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1505: oid 9 version 231 is already newer than 231 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 231 (ObjNum 725 snap 177 seq_num 725) dirty exists 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1505: left oid 9 (ObjNum 725 snap 177 seq_num 725) 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1506: finishing write tid 2 to smithi017107884-6 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1508: finishing write tid 1 to smithi017107884-34 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1506: finishing write tid 3 to smithi017107884-6 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1506: finishing write tid 4 to smithi017107884-6 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1506: oid 6 updating version 0 to 176 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1506: oid 6 version 176 is already newer than 176 2023-03-29T14:08:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1506: oid 6 version 176 is already newer than 176 2023-03-29T14:08:39.468 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 176 (ObjNum 726 snap 177 seq_num 726) dirty exists 2023-03-29T14:08:39.468 INFO:tasks.rados.rados.0.smithi017.stdout:1506: left oid 6 (ObjNum 726 snap 177 seq_num 726) 2023-03-29T14:08:39.468 INFO:tasks.rados.rados.0.smithi017.stdout:1507: expect (ObjNum 694 snap 167 seq_num 694) 2023-03-29T14:08:39.695 INFO:tasks.rados.rados.0.smithi017.stdout:1508: finishing write tid 2 to smithi017107884-34 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1508: finishing write tid 3 to smithi017107884-34 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1508: finishing write tid 4 to smithi017107884-34 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1508: oid 34 updating version 0 to 204 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1508: oid 34 version 204 is already newer than 202 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1508: oid 34 version 204 is already newer than 204 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 34 v 204 (ObjNum 727 snap 177 seq_num 727) dirty exists 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1508: left oid 34 (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1503: done (9 left) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1504: done (8 left) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1505: done (7 left) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1506: done (6 left) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1507: done (5 left) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1508: done (4 left) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1513: done (3 left) 2023-03-29T14:08:39.696 INFO:tasks.rados.rados.0.smithi017.stdout:1514: write oid 14 current snap is 177 2023-03-29T14:08:39.697 INFO:tasks.rados.rados.0.smithi017.stdout:1510: finishing write tid 1 to smithi017107884-49 2023-03-29T14:08:39.697 INFO:tasks.rados.rados.0.smithi017.stdout:1514: seq_num 731 ranges {685917=576907,1893043=392874} 2023-03-29T14:08:39.702 INFO:tasks.rados.rados.0.smithi017.stdout:1514: writing smithi017107884-14 from 685917 to 1262824 tid 1 2023-03-29T14:08:39.708 INFO:tasks.rados.rados.0.smithi017.stdout:1514: writing smithi017107884-14 from 1893043 to 2285917 tid 2 2023-03-29T14:08:39.708 INFO:tasks.rados.rados.0.smithi017.stdout:1515: delete oid 2 current snap is 177 2023-03-29T14:08:39.708 INFO:tasks.rados.rados.0.smithi017.stdout:1511: expect (ObjNum 699 snap 168 seq_num 699) 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1510: finishing write tid 2 to smithi017107884-49 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1510: finishing write tid 3 to smithi017107884-49 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1510: finishing write tid 4 to smithi017107884-49 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1510: oid 49 updating version 0 to 315 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1510: oid 49 version 315 is already newer than 315 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1510: oid 49 version 315 is already newer than 315 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 315 (ObjNum 728 snap 177 seq_num 728) dirty exists 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1510: left oid 49 (ObjNum 728 snap 177 seq_num 728) 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1512: finishing write tid 1 to smithi017107884-30 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1512: finishing write tid 2 to smithi017107884-30 2023-03-29T14:08:40.021 INFO:tasks.rados.rados.0.smithi017.stdout:1512: finishing write tid 3 to smithi017107884-30 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1512: finishing write tid 4 to smithi017107884-30 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1512: oid 30 updating version 0 to 91 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1512: oid 30 version 91 is already newer than 91 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1512: oid 30 version 91 is already newer than 91 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 91 (ObjNum 729 snap 177 seq_num 729) dirty exists 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1512: left oid 30 (ObjNum 729 snap 177 seq_num 729) 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1510: done (4 left) 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1511: done (3 left) 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1512: done (2 left) 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1515: done (1 left) 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1516: read oid 4 snap -1 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1516: expect (ObjNum 610 snap 144 seq_num 610) 2023-03-29T14:08:40.022 INFO:tasks.rados.rados.0.smithi017.stdout:1517: snap_remove snap 170 2023-03-29T14:08:40.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:40.221+0000 7fec2ff6e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:40.226 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:40.226+0000 7fec2ff6e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:40.287 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:40.290 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:40.291 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:40.309 INFO:tasks.rados.rados.0.smithi017.stdout:1517: done (2 left) 2023-03-29T14:08:40.309 INFO:tasks.rados.rados.0.smithi017.stdout:1518: read oid 43 snap -1 2023-03-29T14:08:40.309 INFO:tasks.rados.rados.0.smithi017.stdout:1518: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:40.309 INFO:tasks.rados.rados.0.smithi017.stdout:1519: read oid 18 snap -1 2023-03-29T14:08:40.310 INFO:tasks.rados.rados.0.smithi017.stdout:1519: expect deleted 2023-03-29T14:08:40.310 INFO:tasks.rados.rados.0.smithi017.stdout:1520: write oid 16 current snap is 177 2023-03-29T14:08:40.310 INFO:tasks.rados.rados.0.smithi017.stdout:1514: finishing write tid 1 to smithi017107884-14 2023-03-29T14:08:40.310 INFO:tasks.rados.rados.0.smithi017.stdout:1520: seq_num 733 ranges {719531=727405,1893654=425877} 2023-03-29T14:08:40.317 INFO:tasks.rados.rados.0.smithi017.stdout:1520: writing smithi017107884-16 from 719531 to 1446936 tid 1 2023-03-29T14:08:40.322 INFO:tasks.rados.rados.0.smithi017.stdout:1520: writing smithi017107884-16 from 1893654 to 2319531 tid 2 2023-03-29T14:08:40.322 INFO:tasks.rados.rados.0.smithi017.stdout:1521: write (excl) oid 15 current snap is 177 2023-03-29T14:08:40.322 INFO:tasks.rados.rados.0.smithi017.stdout:1514: finishing write tid 2 to smithi017107884-14 2023-03-29T14:08:40.322 INFO:tasks.rados.rados.0.smithi017.stdout:1514: finishing write tid 3 to smithi017107884-14 2023-03-29T14:08:40.322 INFO:tasks.rados.rados.0.smithi017.stdout:1514: finishing write tid 4 to smithi017107884-14 2023-03-29T14:08:40.322 INFO:tasks.rados.rados.0.smithi017.stdout:1514: oid 14 updating version 0 to 127 2023-03-29T14:08:40.323 INFO:tasks.rados.rados.0.smithi017.stdout:1514: oid 14 version 127 is already newer than 127 2023-03-29T14:08:40.323 INFO:tasks.rados.rados.0.smithi017.stdout:1514: oid 14 version 127 is already newer than 127 2023-03-29T14:08:40.323 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 127 (ObjNum 731 snap 177 seq_num 731) dirty exists 2023-03-29T14:08:40.323 INFO:tasks.rados.rados.0.smithi017.stdout:1514: left oid 14 (ObjNum 731 snap 177 seq_num 731) 2023-03-29T14:08:40.323 INFO:tasks.rados.rados.0.smithi017.stdout:1521: seq_num 734 ranges {736338=402654,1552136=539580} 2023-03-29T14:08:40.328 INFO:tasks.rados.rados.0.smithi017.stdout:1521: writing smithi017107884-15 from 736338 to 1138992 tid 1 2023-03-29T14:08:40.334 INFO:tasks.rados.rados.0.smithi017.stdout:1521: writing smithi017107884-15 from 1552136 to 2091716 tid 2 2023-03-29T14:08:40.334 INFO:tasks.rados.rados.0.smithi017.stdout:1514: done (5 left) 2023-03-29T14:08:40.334 INFO:tasks.rados.rados.0.smithi017.stdout:1522: write (excl) oid 27 current snap is 177 2023-03-29T14:08:40.334 INFO:tasks.rados.rados.0.smithi017.stdout:1522: seq_num 735 ranges {753145=477903,2010618=342527} 2023-03-29T14:08:40.340 INFO:tasks.rados.rados.0.smithi017.stdout:1522: writing smithi017107884-27 from 753145 to 1231048 tid 1 2023-03-29T14:08:40.347 INFO:tasks.rados.rados.0.smithi017.stdout:1522: writing smithi017107884-27 from 2010618 to 2353145 tid 2 2023-03-29T14:08:40.347 INFO:tasks.rados.rados.0.smithi017.stdout:1523: snap_create 2023-03-29T14:08:40.347 INFO:tasks.rados.rados.0.smithi017.stdout:1516: expect (ObjNum 610 snap 144 seq_num 610) 2023-03-29T14:08:40.484 INFO:tasks.rados.rados.0.smithi017.stdout:1518: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1522: finishing write tid 1 to smithi017107884-27 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1521: finishing write tid 1 to smithi017107884-15 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1520: finishing write tid 1 to smithi017107884-16 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1520: finishing write tid 2 to smithi017107884-16 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1520: finishing write tid 3 to smithi017107884-16 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1521: finishing write tid 2 to smithi017107884-15 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1520: finishing write tid 4 to smithi017107884-16 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1520: oid 16 updating version 0 to 116 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1520: oid 16 version 116 is already newer than 116 2023-03-29T14:08:40.612 INFO:tasks.rados.rados.0.smithi017.stdout:1520: oid 16 version 116 is already newer than 116 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 116 (ObjNum 733 snap 177 seq_num 733) dirty exists 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1520: left oid 16 (ObjNum 733 snap 177 seq_num 733) 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1521: finishing write tid 3 to smithi017107884-15 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1521: finishing write tid 4 to smithi017107884-15 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1521: oid 15 updating version 0 to 318 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1521: oid 15 updating version 318 to 321 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1521: oid 15 version 321 is already newer than 321 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 321 (ObjNum 734 snap 177 seq_num 734) dirty exists 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1521: left oid 15 (ObjNum 734 snap 177 seq_num 734) 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1522: finishing write tid 2 to smithi017107884-27 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1522: finishing write tid 3 to smithi017107884-27 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1522: finishing write tid 4 to smithi017107884-27 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1522: oid 27 updating version 0 to 180 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1522: oid 27 version 180 is already newer than 177 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1522: oid 27 version 180 is already newer than 180 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 180 (ObjNum 735 snap 177 seq_num 735) dirty exists 2023-03-29T14:08:40.613 INFO:tasks.rados.rados.0.smithi017.stdout:1522: left oid 27 (ObjNum 735 snap 177 seq_num 735) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1516: done (6 left) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1518: done (5 left) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1519: done (4 left) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1520: done (3 left) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1521: done (2 left) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1522: done (1 left) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1523: done (0 left) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1524: read oid 3 snap -1 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1524: expect (ObjNum 710 snap 175 seq_num 710) 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1525: write oid 37 current snap is 178 2023-03-29T14:08:41.313 INFO:tasks.rados.rados.0.smithi017.stdout:1525: seq_num 736 ranges {769952=553152,2069100=300852} 2023-03-29T14:08:41.320 INFO:tasks.rados.rados.0.smithi017.stdout:1525: writing smithi017107884-37 from 769952 to 1323104 tid 1 2023-03-29T14:08:41.326 INFO:tasks.rados.rados.0.smithi017.stdout:1525: writing smithi017107884-37 from 2069100 to 2369952 tid 2 2023-03-29T14:08:41.326 INFO:tasks.rados.rados.0.smithi017.stdout:1526: snap_remove snap 172 2023-03-29T14:08:42.292 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1526: done (2 left) 2023-03-29T14:08:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1527: read oid 45 snap -1 2023-03-29T14:08:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1527: expect (ObjNum 715 snap 176 seq_num 715) 2023-03-29T14:08:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1528: read oid 6 snap 174 2023-03-29T14:08:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1528: expect (ObjNum 670 snap 164 seq_num 670) 2023-03-29T14:08:42.317 INFO:tasks.rados.rados.0.smithi017.stdout:1529: snap_remove snap 175 2023-03-29T14:08:42.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:42.427+0000 7f8a9ef22700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:42.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:42.432+0000 7f8a9ef22700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:42.494 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:42.497 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:42.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:43.251 INFO:tasks.rados.rados.0.smithi017.stdout:1529: done (4 left) 2023-03-29T14:08:43.252 INFO:tasks.rados.rados.0.smithi017.stdout:1530: write (excl) oid 12 current snap is 178 2023-03-29T14:08:43.252 INFO:tasks.rados.rados.0.smithi017.stdout:1530: seq_num 737 ranges {786759=628401,2011229=375530} 2023-03-29T14:08:43.262 INFO:tasks.rados.rados.0.smithi017.stdout:1530: writing smithi017107884-12 from 786759 to 1415160 tid 1 2023-03-29T14:08:43.269 INFO:tasks.rados.rados.0.smithi017.stdout:1530: writing smithi017107884-12 from 2011229 to 2386759 tid 2 2023-03-29T14:08:43.269 INFO:tasks.rados.rados.0.smithi017.stdout:1531: write oid 31 current snap is 178 2023-03-29T14:08:43.269 INFO:tasks.rados.rados.0.smithi017.stdout:1525: finishing write tid 1 to smithi017107884-37 2023-03-29T14:08:43.269 INFO:tasks.rados.rados.0.smithi017.stdout:1531: seq_num 738 ranges {403566=420003,1386064=546918} 2023-03-29T14:08:43.273 INFO:tasks.rados.rados.0.smithi017.stdout:1531: writing smithi017107884-31 from 403566 to 823569 tid 1 2023-03-29T14:08:43.280 INFO:tasks.rados.rados.0.smithi017.stdout:1531: writing smithi017107884-31 from 1386064 to 1932982 tid 2 2023-03-29T14:08:43.280 INFO:tasks.rados.rados.0.smithi017.stdout:1532: write (excl) oid 19 current snap is 178 2023-03-29T14:08:43.280 INFO:tasks.rados.rados.0.smithi017.stdout:1532: seq_num 739 ranges {420373=495252,1444546=690576} 2023-03-29T14:08:43.285 INFO:tasks.rados.rados.0.smithi017.stdout:1532: writing smithi017107884-19 from 420373 to 915625 tid 1 2023-03-29T14:08:43.292 INFO:tasks.rados.rados.0.smithi017.stdout:1532: writing smithi017107884-19 from 1444546 to 2135122 tid 2 2023-03-29T14:08:43.292 INFO:tasks.rados.rados.0.smithi017.stdout:1533: snap_create 2023-03-29T14:08:43.292 INFO:tasks.rados.rados.0.smithi017.stdout:1525: finishing write tid 2 to smithi017107884-37 2023-03-29T14:08:43.292 INFO:tasks.rados.rados.0.smithi017.stdout:1525: finishing write tid 3 to smithi017107884-37 2023-03-29T14:08:43.292 INFO:tasks.rados.rados.0.smithi017.stdout:1524: expect (ObjNum 710 snap 175 seq_num 710) 2023-03-29T14:08:43.540 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:43.540 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:43.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:43.551 INFO:tasks.rados.rados.0.smithi017.stdout:1525: finishing write tid 4 to smithi017107884-37 2023-03-29T14:08:43.551 INFO:tasks.rados.rados.0.smithi017.stdout:1525: oid 37 updating version 0 to 94 2023-03-29T14:08:43.551 INFO:tasks.rados.rados.0.smithi017.stdout:1525: oid 37 version 94 is already newer than 94 2023-03-29T14:08:43.551 INFO:tasks.rados.rados.0.smithi017.stdout:1525: oid 37 version 94 is already newer than 94 2023-03-29T14:08:43.551 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 94 (ObjNum 736 snap 178 seq_num 736) dirty exists 2023-03-29T14:08:43.551 INFO:tasks.rados.rados.0.smithi017.stdout:1525: left oid 37 (ObjNum 736 snap 178 seq_num 736) 2023-03-29T14:08:43.551 INFO:tasks.rados.rados.0.smithi017.stdout:1528: expect (ObjNum 670 snap 164 seq_num 670) 2023-03-29T14:08:43.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:43.668+0000 7f633c2b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:43.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:43.675+0000 7f633c2b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:43.747 INFO:tasks.rados.rados.0.smithi017.stdout:1527: expect (ObjNum 715 snap 176 seq_num 715) 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1531: finishing write tid 1 to smithi017107884-31 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1532: finishing write tid 1 to smithi017107884-19 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1531: finishing write tid 2 to smithi017107884-31 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1530: finishing write tid 1 to smithi017107884-12 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1531: finishing write tid 3 to smithi017107884-31 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1531: finishing write tid 4 to smithi017107884-31 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1531: oid 31 updating version 0 to 131 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1531: oid 31 version 131 is already newer than 131 2023-03-29T14:08:43.824 INFO:tasks.rados.rados.0.smithi017.stdout:1531: oid 31 version 131 is already newer than 131 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 131 (ObjNum 738 snap 178 seq_num 738) dirty exists 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1531: left oid 31 (ObjNum 738 snap 178 seq_num 738) 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1530: finishing write tid 2 to smithi017107884-12 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1530: finishing write tid 3 to smithi017107884-12 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1530: finishing write tid 4 to smithi017107884-12 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1530: oid 12 updating version 0 to 220 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1530: oid 12 version 220 is already newer than 217 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1530: oid 12 version 220 is already newer than 220 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 12 v 220 (ObjNum 737 snap 178 seq_num 737) dirty exists 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1530: left oid 12 (ObjNum 737 snap 178 seq_num 737) 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1532: finishing write tid 2 to smithi017107884-19 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1532: finishing write tid 3 to smithi017107884-19 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1532: finishing write tid 4 to smithi017107884-19 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1532: oid 19 updating version 0 to 204 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1532: oid 19 updating version 204 to 207 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1532: oid 19 version 207 is already newer than 207 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 207 (ObjNum 739 snap 178 seq_num 739) dirty exists 2023-03-29T14:08:43.825 INFO:tasks.rados.rados.0.smithi017.stdout:1532: left oid 19 (ObjNum 739 snap 178 seq_num 739) 2023-03-29T14:08:43.878 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:43.880 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:43.891 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":568,"stamp":"2023-03-29T14:08:43.250111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235212492,"num_objects":834,"num_object_clones":738,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3399,"num_read_kb":1973571,"num_write":4712,"num_write_kb":1293642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2486,"ondisk_log_size":2486,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1639112,"kb_used_data":1559016,"kb_used_omap":0,"kb_used_meta":80064,"kb_avail":375848248,"statfs":{"total":386547056640,"available":384868605952,"internally_reserved":0,"allocated":1596432384,"data_stored":1954853716,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":81985536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19091300,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-4,"num_read":78,"num_read_kb":52617,"num_write":104,"num_write_kb":25144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.948349"},"pg_stats":[{"pgid":"3.f","version":"373'116","reported_seq":350,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:40.481496+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:40.481496+0000","last_peered":"2023-03-29T14:08:40.481496+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:40.481496+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12336932,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":65943,"num_write":146,"num_write_kb":42272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"361'44","reported_seq":382,"reported_epoch":373,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:39.380064+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:39.380064+0000","last_peered":"2023-03-29T14:08:39.380064+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:39.380064+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7563414,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":25432,"num_write":82,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"365'237","reported_seq":844,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-03-29T14:08:40.333375+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:40.333375+0000","last_peered":"2023-03-29T14:08:40.333375+0000","last_clean":"2023-03-29T14:08:40.333375+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:40.333375+0000","last_undegraded":"2023-03-29T14:08:40.333375+0000","last_fullsized":"2023-03-29T14:08:40.333375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32976654,"num_objects":84,"num_object_clones":74,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":348,"num_read_kb":243942,"num_write":458,"num_write_kb":136364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"371'231","reported_seq":829,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-03-29T14:08:40.330436+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:40.330436+0000","last_peered":"2023-03-29T14:08:40.330436+0000","last_clean":"2023-03-29T14:08:40.330436+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:40.330436+0000","last_undegraded":"2023-03-29T14:08:40.330436+0000","last_fullsized":"2023-03-29T14:08:40.330436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23497374,"num_objects":90,"num_object_clones":82,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":335,"num_read_kb":206400,"num_write":452,"num_write_kb":128762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"371'204","reported_seq":738,"reported_epoch":373,"state":"active+clean","last_fresh":"2023-03-29T14:08:39.454299+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:39.454299+0000","last_peered":"2023-03-29T14:08:39.454299+0000","last_clean":"2023-03-29T14:08:39.454299+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:39.454299+0000","last_undegraded":"2023-03-29T14:08:39.454299+0000","last_fullsized":"2023-03-29T14:08:39.454299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21850288,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":263,"num_read_kb":155481,"num_write":408,"num_write_kb":119746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":314,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-03-29T14:08:40.322572+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:40.322572+0000","last_peered":"2023-03-29T14:08:40.322572+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:40.322572+0000","last_undegraded":"2023-03-29T14:08:40.322572+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"372'127","reported_seq":580,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:40.325729+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:40.325729+0000","last_peered":"2023-03-29T14:08:40.325729+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:40.325729+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8639458,"num_objects":44,"num_object_clones":38,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":147,"num_read_kb":67877,"num_write":230,"num_write_kb":65316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":44}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":488,"reported_epoch":374,"state":"active+clean","last_fresh":"2023-03-29T14:08:40.332128+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:40.332128+0000","last_peered":"2023-03-29T14:08:40.332128+0000","last_clean":"2023-03-29T14:08:40.332128+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:40.332128+0000","last_undegraded":"2023-03-29T14:08:40.332128+0000","last_fullsized":"2023-03-29T14:08:40.332128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":293,"reported_epoch":373,"state":"active+undersized","last_fresh":"2023-03-29T14:08:39.357785+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:39.357785+0000","last_peered":"2023-03-29T14:08:39.357785+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:39.357785+0000","last_undegraded":"2023-03-29T14:08:39.357785+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"372'91","reported_seq":264,"reported_epoch":373,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:39.497684+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:39.497684+0000","last_peered":"2023-03-29T14:08:39.497684+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:39.497684+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4504606,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":38,"num_read_kb":12594,"num_write":104,"num_write_kb":27516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":317,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-03-29T14:08:40.334702+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:40.334702+0000","last_peered":"2023-03-29T14:08:40.334702+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:40.334702+0000","last_undegraded":"2023-03-29T14:08:40.334702+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"361'93","reported_seq":512,"reported_epoch":373,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:39.363674+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:39.363674+0000","last_peered":"2023-03-29T14:08:39.363674+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:39.363674+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731484,"num_objects":38,"num_object_clones":34,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":149,"num_read_kb":100906,"num_write":182,"num_write_kb":52894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":373,"state":"active+undersized","last_fresh":"2023-03-29T14:08:39.348083+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:39.348083+0000","last_peered":"2023-03-29T14:08:39.348083+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:39.348083+0000","last_undegraded":"2023-03-29T14:08:39.348083+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"368'131","reported_seq":590,"reported_epoch":373,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:40.422079+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:40.422079+0000","last_peered":"2023-03-29T14:08:40.422079+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:40.422079+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10925078,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":202,"num_read_kb":118559,"num_write":252,"num_write_kb":74172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":317,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-03-29T14:08:40.320587+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:40.320587+0000","last_peered":"2023-03-29T14:08:40.320587+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:40.320587+0000","last_undegraded":"2023-03-29T14:08:40.320587+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":518,"reported_epoch":373,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:39.377604+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:39.377604+0000","last_peered":"2023-03-29T14:08:39.377604+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:39.377604+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"373'180","reported_seq":759,"reported_epoch":374,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:40.493900+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:40.493900+0000","last_peered":"2023-03-29T14:08:40.493900+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:40.493900+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043222,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":332,"num_read_kb":224431,"num_write":350,"num_write_kb":101254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":64}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":317,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-03-29T14:08:40.319743+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:40.319743+0000","last_peered":"2023-03-29T14:08:40.319743+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:40.319743+0000","last_undegraded":"2023-03-29T14:08:40.319743+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"373'321","reported_seq":1021,"reported_epoch":373,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:40.470276+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:40.470276+0000","last_peered":"2023-03-29T14:08:40.470276+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:40.470276+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254448,"num_objects":112,"num_object_clones":98,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":482,"num_read_kb":284964,"num_write":604,"num_write_kb":174738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":112}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":373,"state":"active+undersized","last_fresh":"2023-03-29T14:08:39.395354+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:39.395354+0000","last_peered":"2023-03-29T14:08:39.395354+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:39.395354+0000","last_undegraded":"2023-03-29T14:08:39.395354+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"372'116","reported_seq":539,"reported_epoch":373,"state":"active+clean","last_fresh":"2023-03-29T14:08:39.411279+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:39.411279+0000","last_peered":"2023-03-29T14:08:39.411279+0000","last_clean":"2023-03-29T14:08:39.411279+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:39.411279+0000","last_undegraded":"2023-03-29T14:08:39.411279+0000","last_fullsized":"2023-03-29T14:08:39.411279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":42,"num_object_clones":38,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":79797,"num_write":234,"num_write_kb":68058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"371'216","reported_seq":749,"reported_epoch":373,"state":"active+clean","last_fresh":"2023-03-29T14:08:39.385030+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:39.385030+0000","last_peered":"2023-03-29T14:08:39.385030+0000","last_clean":"2023-03-29T14:08:39.385030+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:39.385030+0000","last_undegraded":"2023-03-29T14:08:39.385030+0000","last_fullsized":"2023-03-29T14:08:39.385030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16202872,"num_objects":76,"num_object_clones":66,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":268,"num_read_kb":159208,"num_write":396,"num_write_kb":115454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":293,"reported_epoch":373,"state":"active+undersized","last_fresh":"2023-03-29T14:08:39.386829+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:39.386829+0000","last_peered":"2023-03-29T14:08:39.386829+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:39.386829+0000","last_undegraded":"2023-03-29T14:08:39.386829+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":317,"reported_epoch":374,"state":"active+undersized","last_fresh":"2023-03-29T14:08:40.333898+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:40.333898+0000","last_peered":"2023-03-29T14:08:40.333898+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:40.333898+0000","last_undegraded":"2023-03-29T14:08:40.333898+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"371'203","reported_seq":747,"reported_epoch":373,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:39.368433+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:39.368433+0000","last_peered":"2023-03-29T14:08:39.368433+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:39.368433+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950602,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":279,"num_read_kb":169950,"num_write":388,"num_write_kb":114584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":72}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":234031756,"num_objects":830,"num_object_clones":738,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3257,"num_read_kb":1973451,"num_write":4438,"num_write_kb":1289854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2389,"ondisk_log_size":2389,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":477992,"kb_used_data":455840,"kb_used_omap":0,"kb_used_meta":22144,"kb_avail":93893848,"statfs":{"total":96636764160,"available":96147300352,"internally_reserved":0,"allocated":466780160,"data_stored":573373961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22675456},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":529240,"kb_used_data":505872,"kb_used_omap":0,"kb_used_meta":23360,"kb_avail":93842600,"statfs":{"total":96636764160,"available":96094822400,"internally_reserved":0,"allocated":518012928,"data_stored":637816908,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23920640},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":475936,"kb_used_data":453464,"kb_used_omap":0,"kb_used_meta":22464,"kb_avail":93895904,"statfs":{"total":96636764160,"available":96149405696,"internally_reserved":0,"allocated":464347136,"data_stored":570430622,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23003136},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:43.895 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.3 3.0 1.0 3.2 3.1 2023-03-29T14:08:44.070 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:44.060+0000 7f74cb1d1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:44.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:44.074+0000 7f74cb1d1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:44.255 INFO:tasks.rados.rados.0.smithi017.stdout:1524: done (7 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1525: done (6 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1527: done (5 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1528: done (4 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1530: done (3 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1531: done (2 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1532: done (1 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1533: done (0 left) 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1534: write oid 14 current snap is 179 2023-03-29T14:08:44.256 INFO:tasks.rados.rados.0.smithi017.stdout:1534: seq_num 740 ranges {437180=570501,1503028=550587} 2023-03-29T14:08:44.261 INFO:tasks.rados.rados.0.smithi017.stdout:1534: writing smithi017107884-14 from 437180 to 1007681 tid 1 2023-03-29T14:08:44.267 INFO:tasks.rados.rados.0.smithi017.stdout:1534: writing smithi017107884-14 from 1503028 to 2053615 tid 2 2023-03-29T14:08:44.267 INFO:tasks.rados.rados.0.smithi017.stdout:1535: snap_create 2023-03-29T14:08:44.282 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3] on osd.0 to force-recovery; instructing pg(s) [1.0,3.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.2,3.f] on osd.3 to force-recovery; 2023-03-29T14:08:44.282 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:08:44.316 INFO:tasks.rados.rados.0.smithi017.stdout:1534: finishing write tid 1 to smithi017107884-14 2023-03-29T14:08:44.362 INFO:tasks.rados.rados.0.smithi017.stdout:1534: finishing write tid 2 to smithi017107884-14 2023-03-29T14:08:44.363 INFO:tasks.rados.rados.0.smithi017.stdout:1534: finishing write tid 3 to smithi017107884-14 2023-03-29T14:08:44.364 INFO:tasks.rados.rados.0.smithi017.stdout:1534: finishing write tid 4 to smithi017107884-14 2023-03-29T14:08:44.364 INFO:tasks.rados.rados.0.smithi017.stdout:1534: oid 14 updating version 0 to 135 2023-03-29T14:08:44.364 INFO:tasks.rados.rados.0.smithi017.stdout:1534: oid 14 version 135 is already newer than 135 2023-03-29T14:08:44.364 INFO:tasks.rados.rados.0.smithi017.stdout:1534: oid 14 version 135 is already newer than 135 2023-03-29T14:08:44.364 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 135 (ObjNum 740 snap 179 seq_num 740) dirty exists 2023-03-29T14:08:44.364 INFO:tasks.rados.rados.0.smithi017.stdout:1534: left oid 14 (ObjNum 740 snap 179 seq_num 740) 2023-03-29T14:08:44.498 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:44.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:44.625+0000 7f7c57271700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:44.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:44.630+0000 7f7c57271700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:44.693 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:44.693 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:44.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~215s 2023-03-29T14:08:45.256 INFO:tasks.rados.rados.0.smithi017.stdout:1534: done (1 left) 2023-03-29T14:08:45.256 INFO:tasks.rados.rados.0.smithi017.stdout:1535: done (0 left) 2023-03-29T14:08:45.256 INFO:tasks.rados.rados.0.smithi017.stdout:1536: write oid 42 current snap is 180 2023-03-29T14:08:45.256 INFO:tasks.rados.rados.0.smithi017.stdout:1536: seq_num 741 ranges {453987=645750,1845157=608830} 2023-03-29T14:08:45.262 INFO:tasks.rados.rados.0.smithi017.stdout:1536: writing smithi017107884-42 from 453987 to 1099737 tid 1 2023-03-29T14:08:45.269 INFO:tasks.rados.rados.0.smithi017.stdout:1536: writing smithi017107884-42 from 1845157 to 2453987 tid 2 2023-03-29T14:08:45.269 INFO:tasks.rados.rados.0.smithi017.stdout:1537: snap_remove snap 177 2023-03-29T14:08:46.262 INFO:tasks.rados.rados.0.smithi017.stdout:1537: done (1 left) 2023-03-29T14:08:46.262 INFO:tasks.rados.rados.0.smithi017.stdout:1538: read oid 39 snap -1 2023-03-29T14:08:46.262 INFO:tasks.rados.rados.0.smithi017.stdout:1538: expect (ObjNum 673 snap 164 seq_num 673) 2023-03-29T14:08:46.262 INFO:tasks.rados.rados.0.smithi017.stdout:1539: write oid 25 current snap is 180 2023-03-29T14:08:46.262 INFO:tasks.rados.rados.0.smithi017.stdout:1536: finishing write tid 1 to smithi017107884-42 2023-03-29T14:08:46.262 INFO:tasks.rados.rados.0.smithi017.stdout:1539: seq_num 742 ranges {470794=720999,1903639=554256} 2023-03-29T14:08:46.268 INFO:tasks.rados.rados.0.smithi017.stdout:1539: writing smithi017107884-25 from 470794 to 1191793 tid 1 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1539: writing smithi017107884-25 from 1903639 to 2457895 tid 2 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1540: read oid 43 snap -1 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1540: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1541: snap_create 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1536: finishing write tid 2 to smithi017107884-42 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1536: finishing write tid 3 to smithi017107884-42 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1536: finishing write tid 4 to smithi017107884-42 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1536: oid 42 updating version 0 to 235 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1536: oid 42 version 235 is already newer than 235 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1536: oid 42 version 235 is already newer than 235 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 235 (ObjNum 741 snap 180 seq_num 741) dirty exists 2023-03-29T14:08:46.276 INFO:tasks.rados.rados.0.smithi017.stdout:1536: left oid 42 (ObjNum 741 snap 180 seq_num 741) 2023-03-29T14:08:46.325 INFO:tasks.rados.rados.0.smithi017.stdout:1538: expect (ObjNum 673 snap 164 seq_num 673) 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:1539: finishing write tid 1 to smithi017107884-25 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:1539: finishing write tid 2 to smithi017107884-25 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:1539: finishing write tid 3 to smithi017107884-25 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:1539: finishing write tid 4 to smithi017107884-25 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:1539: oid 25 updating version 0 to 239 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:1539: oid 25 version 239 is already newer than 239 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:1539: oid 25 version 239 is already newer than 239 2023-03-29T14:08:46.540 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 239 (ObjNum 742 snap 180 seq_num 742) dirty exists 2023-03-29T14:08:46.541 INFO:tasks.rados.rados.0.smithi017.stdout:1539: left oid 25 (ObjNum 742 snap 180 seq_num 742) 2023-03-29T14:08:46.541 INFO:tasks.rados.rados.0.smithi017.stdout:1540: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:46.694 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:46.822+0000 7f40037b8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:46.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:46.826+0000 7f40037b8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:46.887 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:46.890 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:46.891 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:47.263 INFO:tasks.rados.rados.0.smithi017.stdout:1536: done (4 left) 2023-03-29T14:08:47.263 INFO:tasks.rados.rados.0.smithi017.stdout:1538: done (3 left) 2023-03-29T14:08:47.264 INFO:tasks.rados.rados.0.smithi017.stdout:1539: done (2 left) 2023-03-29T14:08:47.264 INFO:tasks.rados.rados.0.smithi017.stdout:1540: done (1 left) 2023-03-29T14:08:47.264 INFO:tasks.rados.rados.0.smithi017.stdout:1541: done (0 left) 2023-03-29T14:08:47.264 INFO:tasks.rados.rados.0.smithi017.stdout:1542: delete oid 13 current snap is 181 2023-03-29T14:08:47.343 INFO:tasks.rados.rados.0.smithi017.stdout:1542: done (0 left) 2023-03-29T14:08:47.343 INFO:tasks.rados.rados.0.smithi017.stdout:1543: write (excl) oid 10 current snap is 181 2023-03-29T14:08:47.343 INFO:tasks.rados.rados.0.smithi017.stdout:1543: seq_num 744 ranges {504408=471497,1620603=557925} 2023-03-29T14:08:47.348 INFO:tasks.rados.rados.0.smithi017.stdout:1543: writing smithi017107884-10 from 504408 to 975905 tid 1 2023-03-29T14:08:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1543: writing smithi017107884-10 from 1620603 to 2178528 tid 2 2023-03-29T14:08:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1544: write (excl) oid 1 current snap is 181 2023-03-29T14:08:47.355 INFO:tasks.rados.rados.0.smithi017.stdout:1544: seq_num 745 ranges {521215=546746,1562732=701583} 2023-03-29T14:08:47.360 INFO:tasks.rados.rados.0.smithi017.stdout:1544: writing smithi017107884-1 from 521215 to 1067961 tid 1 2023-03-29T14:08:47.367 INFO:tasks.rados.rados.0.smithi017.stdout:1544: writing smithi017107884-1 from 1562732 to 2264315 tid 2 2023-03-29T14:08:47.367 INFO:tasks.rados.rados.0.smithi017.stdout:1545: read oid 29 snap -1 2023-03-29T14:08:47.367 INFO:tasks.rados.rados.0.smithi017.stdout:1545: expect (ObjNum 706 snap 170 seq_num 706) 2023-03-29T14:08:47.367 INFO:tasks.rados.rados.0.smithi017.stdout:1546: snap_create 2023-03-29T14:08:47.428 INFO:tasks.rados.rados.0.smithi017.stdout:1543: finishing write tid 1 to smithi017107884-10 2023-03-29T14:08:47.429 INFO:tasks.rados.rados.0.smithi017.stdout:1544: finishing write tid 1 to smithi017107884-1 2023-03-29T14:08:47.432 INFO:tasks.rados.rados.0.smithi017.stdout:1545: expect (ObjNum 706 snap 170 seq_num 706) 2023-03-29T14:08:47.671 INFO:tasks.rados.rados.0.smithi017.stdout:1543: finishing write tid 2 to smithi017107884-10 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1543: finishing write tid 3 to smithi017107884-10 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1543: finishing write tid 4 to smithi017107884-10 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1543: oid 10 updating version 0 to 241 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1543: oid 10 version 241 is already newer than 238 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1543: oid 10 version 241 is already newer than 241 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 241 (ObjNum 744 snap 181 seq_num 744) dirty exists 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1543: left oid 10 (ObjNum 744 snap 181 seq_num 744) 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1544: finishing write tid 2 to smithi017107884-1 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1544: finishing write tid 3 to smithi017107884-1 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1544: finishing write tid 4 to smithi017107884-1 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1544: oid 1 updating version 0 to 210 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1544: oid 1 version 210 is already newer than 208 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1544: oid 1 version 210 is already newer than 210 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 210 (ObjNum 745 snap 181 seq_num 745) dirty exists 2023-03-29T14:08:47.672 INFO:tasks.rados.rados.0.smithi017.stdout:1544: left oid 1 (ObjNum 745 snap 181 seq_num 745) 2023-03-29T14:08:48.269 INFO:tasks.rados.rados.0.smithi017.stdout:1543: done (3 left) 2023-03-29T14:08:48.269 INFO:tasks.rados.rados.0.smithi017.stdout:1544: done (2 left) 2023-03-29T14:08:48.269 INFO:tasks.rados.rados.0.smithi017.stdout:1545: done (1 left) 2023-03-29T14:08:48.269 INFO:tasks.rados.rados.0.smithi017.stdout:1546: done (0 left) 2023-03-29T14:08:48.269 INFO:tasks.rados.rados.0.smithi017.stdout:1547: snap_create 2023-03-29T14:08:48.892 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:49.020+0000 7f604ee68700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:49.024+0000 7f604ee68700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:49.092 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:49.092 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:49.270 INFO:tasks.rados.rados.0.smithi017.stdout:1547: done (0 left) 2023-03-29T14:08:49.270 INFO:tasks.rados.rados.0.smithi017.stdout:1548: delete oid 29 current snap is 183 2023-03-29T14:08:49.295 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:49.295 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:49.295 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:49.415 INFO:tasks.rados.rados.0.smithi017.stdout:1548: done (0 left) 2023-03-29T14:08:49.415 INFO:tasks.rados.rados.0.smithi017.stdout:1549: read oid 43 snap -1 2023-03-29T14:08:49.415 INFO:tasks.rados.rados.0.smithi017.stdout:1549: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:49.415 INFO:tasks.rados.rados.0.smithi017.stdout:1550: write oid 3 current snap is 183 2023-03-29T14:08:49.415 INFO:tasks.rados.rados.0.smithi017.stdout:1550: seq_num 747 ranges {554829=413597,1396049=705252} 2023-03-29T14:08:49.422 INFO:tasks.rados.rados.0.smithi017.stdout:1550: writing smithi017107884-3 from 554829 to 968426 tid 1 2023-03-29T14:08:49.429 INFO:tasks.rados.rados.0.smithi017.stdout:1550: writing smithi017107884-3 from 1396049 to 2101301 tid 2 2023-03-29T14:08:49.429 INFO:tasks.rados.rados.0.smithi017.stdout:1551: read oid 18 snap -1 2023-03-29T14:08:49.429 INFO:tasks.rados.rados.0.smithi017.stdout:1551: expect deleted 2023-03-29T14:08:49.429 INFO:tasks.rados.rados.0.smithi017.stdout:1552: read oid 29 snap -1 2023-03-29T14:08:49.429 INFO:tasks.rados.rados.0.smithi017.stdout:1552: expect deleted 2023-03-29T14:08:49.429 INFO:tasks.rados.rados.0.smithi017.stdout:1553: delete oid 28 current snap is 183 2023-03-29T14:08:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:49.429+0000 7f284c78f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:49.437+0000 7f284c78f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:49.441 INFO:tasks.rados.rados.0.smithi017.stdout:1549: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:08:49.571 INFO:tasks.rados.rados.0.smithi017.stdout:1550: finishing write tid 1 to smithi017107884-3 2023-03-29T14:08:49.571 INFO:tasks.rados.rados.0.smithi017.stdout:1550: finishing write tid 2 to smithi017107884-3 2023-03-29T14:08:49.571 INFO:tasks.rados.rados.0.smithi017.stdout:1549: done (4 left) 2023-03-29T14:08:49.571 INFO:tasks.rados.rados.0.smithi017.stdout:1551: done (3 left) 2023-03-29T14:08:49.571 INFO:tasks.rados.rados.0.smithi017.stdout:1553: done (2 left) 2023-03-29T14:08:49.571 INFO:tasks.rados.rados.0.smithi017.stdout:1554: snap_remove snap 171 2023-03-29T14:08:49.847 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:49.850 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:49.867 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":577,"stamp":"2023-03-29T14:08:49.268761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":236288668,"num_objects":846,"num_object_clones":750,"num_object_copies":2538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":3438,"num_read_kb":2005649,"num_write":4768,"num_write_kb":1308622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2513,"ondisk_log_size":2513,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1655660,"kb_used_data":1575180,"kb_used_omap":0,"kb_used_meta":80448,"kb_avail":375831700,"statfs":{"total":386547056640,"available":384851660800,"internally_reserved":0,"allocated":1612984320,"data_stored":1977568398,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":82378752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1076176,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":39,"num_read_kb":32078,"num_write":56,"num_write_kb":14980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014132"},"pg_stats":[{"pgid":"3.f","version":"373'116","reported_seq":355,"reported_epoch":379,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:45.270951+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:45.270951+0000","last_peered":"2023-03-29T14:08:45.270951+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:45.270951+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12336932,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":65943,"num_write":146,"num_write_kb":42272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"361'44","reported_seq":385,"reported_epoch":379,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:42.375563+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:42.375563+0000","last_peered":"2023-03-29T14:08:42.375563+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:42.375563+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7563414,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":25432,"num_write":82,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"365'237","reported_seq":852,"reported_epoch":379,"state":"active+clean","last_fresh":"2023-03-29T14:08:46.318533+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:46.318533+0000","last_peered":"2023-03-29T14:08:46.318533+0000","last_clean":"2023-03-29T14:08:46.318533+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:46.318533+0000","last_undegraded":"2023-03-29T14:08:46.318533+0000","last_fullsized":"2023-03-29T14:08:46.318533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32976654,"num_objects":84,"num_object_clones":74,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":251012,"num_write":458,"num_write_kb":136364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"379'239","reported_seq":848,"reported_epoch":379,"state":"active+clean","last_fresh":"2023-03-29T14:08:46.316477+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:46.316477+0000","last_peered":"2023-03-29T14:08:46.316477+0000","last_clean":"2023-03-29T14:08:46.316477+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:46.316477+0000","last_undegraded":"2023-03-29T14:08:46.316477+0000","last_fullsized":"2023-03-29T14:08:46.316477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245302,"num_objects":94,"num_object_clones":86,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":337,"num_read_kb":206402,"num_write":468,"num_write_kb":133708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"371'204","reported_seq":744,"reported_epoch":379,"state":"active+clean","last_fresh":"2023-03-29T14:08:42.415511+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:42.415511+0000","last_peered":"2023-03-29T14:08:42.415511+0000","last_clean":"2023-03-29T14:08:42.415511+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:42.415511+0000","last_undegraded":"2023-03-29T14:08:42.415511+0000","last_fullsized":"2023-03-29T14:08:42.415511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21850288,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":270,"num_read_kb":160561,"num_write":408,"num_write_kb":119746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":319,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:45.265599+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:45.265599+0000","last_peered":"2023-03-29T14:08:45.265599+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:45.265599+0000","last_undegraded":"2023-03-29T14:08:45.265599+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"377'135","reported_seq":599,"reported_epoch":379,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:45.272934+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:45.272934+0000","last_peered":"2023-03-29T14:08:45.272934+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:45.272934+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9681492,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":149,"num_read_kb":67879,"num_write":246,"num_write_kb":69400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":48}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":493,"reported_epoch":379,"state":"active+clean","last_fresh":"2023-03-29T14:08:45.266944+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:45.266944+0000","last_peered":"2023-03-29T14:08:45.266944+0000","last_clean":"2023-03-29T14:08:45.266944+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:45.266944+0000","last_undegraded":"2023-03-29T14:08:45.266944+0000","last_fullsized":"2023-03-29T14:08:45.266944+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":110,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":296,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:42.337469+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:42.337469+0000","last_peered":"2023-03-29T14:08:42.337469+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:42.337469+0000","last_undegraded":"2023-03-29T14:08:42.337469+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"374'94","reported_seq":275,"reported_epoch":377,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:43.297963+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:43.297963+0000","last_peered":"2023-03-29T14:08:43.297963+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:43.297963+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9244510,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":12595,"num_write":112,"num_write_kb":29186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":322,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:45.268162+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:45.268162+0000","last_peered":"2023-03-29T14:08:45.268162+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:45.268162+0000","last_undegraded":"2023-03-29T14:08:45.268162+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"361'93","reported_seq":516,"reported_epoch":377,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:43.300971+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:43.300971+0000","last_peered":"2023-03-29T14:08:43.300971+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:43.300971+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731484,"num_objects":38,"num_object_clones":34,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":149,"num_read_kb":100906,"num_write":182,"num_write_kb":52894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":38}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:42.384524+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:42.384524+0000","last_peered":"2023-03-29T14:08:42.384524+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:42.384524+0000","last_undegraded":"2023-03-29T14:08:42.384524+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"368'131","reported_seq":593,"reported_epoch":379,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:42.381488+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:42.381488+0000","last_peered":"2023-03-29T14:08:42.381488+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:42.381488+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10925078,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":202,"num_read_kb":118559,"num_write":252,"num_write_kb":74172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":322,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:45.273695+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:45.273695+0000","last_peered":"2023-03-29T14:08:45.273695+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:45.273695+0000","last_undegraded":"2023-03-29T14:08:45.273695+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":521,"reported_epoch":379,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:42.386454+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:42.386454+0000","last_peered":"2023-03-29T14:08:42.386454+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:42.386454+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"373'180","reported_seq":767,"reported_epoch":379,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:45.275067+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:45.275067+0000","last_peered":"2023-03-29T14:08:45.275067+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:45.275067+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043222,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":339,"num_read_kb":232829,"num_write":350,"num_write_kb":101254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":64}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":322,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:45.267472+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:45.267472+0000","last_peered":"2023-03-29T14:08:45.267472+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:45.267472+0000","last_undegraded":"2023-03-29T14:08:45.267472+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"373'321","reported_seq":1028,"reported_epoch":377,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:43.305614+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:43.305614+0000","last_peered":"2023-03-29T14:08:43.305614+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:43.305614+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254448,"num_objects":112,"num_object_clones":98,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":489,"num_read_kb":296487,"num_write":604,"num_write_kb":174738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":112}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":296,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:42.374235+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:42.374235+0000","last_peered":"2023-03-29T14:08:42.374235+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:42.374235+0000","last_undegraded":"2023-03-29T14:08:42.374235+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"372'116","reported_seq":542,"reported_epoch":379,"state":"active+clean","last_fresh":"2023-03-29T14:08:42.364583+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:42.364583+0000","last_peered":"2023-03-29T14:08:42.364583+0000","last_clean":"2023-03-29T14:08:42.364583+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:42.364583+0000","last_undegraded":"2023-03-29T14:08:42.364583+0000","last_fullsized":"2023-03-29T14:08:42.364583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":42,"num_object_clones":38,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":79797,"num_write":234,"num_write_kb":68058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"376'220","reported_seq":759,"reported_epoch":379,"state":"active+clean","last_fresh":"2023-03-29T14:08:43.440416+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:43.440416+0000","last_peered":"2023-03-29T14:08:43.440416+0000","last_clean":"2023-03-29T14:08:43.440416+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:43.440416+0000","last_undegraded":"2023-03-29T14:08:43.440416+0000","last_fullsized":"2023-03-29T14:08:43.440416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16774310,"num_objects":78,"num_object_clones":68,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":271,"num_read_kb":159209,"num_write":404,"num_write_kb":117416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":296,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:42.369592+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:42.369592+0000","last_peered":"2023-03-29T14:08:42.369592+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:42.369592+0000","last_undegraded":"2023-03-29T14:08:42.369592+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":322,"reported_epoch":379,"state":"active+undersized","last_fresh":"2023-03-29T14:08:45.275948+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:45.275948+0000","last_peered":"2023-03-29T14:08:45.275948+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:45.275948+0000","last_undegraded":"2023-03-29T14:08:45.275948+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"376'207","reported_seq":758,"reported_epoch":377,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:43.500907+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:43.500907+0000","last_peered":"2023-03-29T14:08:43.500907+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:43.500907+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18925474,"num_objects":74,"num_object_clones":66,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":282,"num_read_kb":169951,"num_write":396,"num_write_kb":116902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":74}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":235107932,"num_objects":842,"num_object_clones":750,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":3296,"num_read_kb":2005529,"num_write":4494,"num_write_kb":1304834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2416,"ondisk_log_size":2416,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":484548,"kb_used_data":462268,"kb_used_omap":0,"kb_used_meta":22272,"kb_avail":93887292,"statfs":{"total":96636764160,"available":96140587008,"internally_reserved":0,"allocated":473362432,"data_stored":581990399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22806528},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":533632,"kb_used_data":510136,"kb_used_omap":0,"kb_used_meta":23488,"kb_avail":93838208,"statfs":{"total":96636764160,"available":96090324992,"internally_reserved":0,"allocated":522379264,"data_stored":643675873,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24051712},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":481536,"kb_used_data":458936,"kb_used_omap":0,"kb_used_meta":22592,"kb_avail":93890304,"statfs":{"total":96636764160,"available":96143671296,"internally_reserved":0,"allocated":469950464,"data_stored":578669901,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23134208},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:50.228 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~220s 2023-03-29T14:08:50.276 INFO:tasks.rados.rados.0.smithi017.stdout:1554: done (2 left) 2023-03-29T14:08:50.276 INFO:tasks.rados.rados.0.smithi017.stdout:1555: snap_remove snap 179 2023-03-29T14:08:51.093 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:51.222 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:51.221+0000 7f52253f3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:51.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:51.225+0000 7f52253f3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:51.280 INFO:tasks.rados.rados.0.smithi017.stdout:1555: done (2 left) 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:1556: delete oid 33 current snap is 183 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:1550: finishing write tid 3 to smithi017107884-3 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:1550: finishing write tid 4 to smithi017107884-3 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:1550: oid 3 updating version 0 to 325 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:1550: oid 3 version 325 is already newer than 325 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:1550: oid 3 version 325 is already newer than 325 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 325 (ObjNum 747 snap 183 seq_num 747) dirty exists 2023-03-29T14:08:51.281 INFO:tasks.rados.rados.0.smithi017.stdout:1550: left oid 3 (ObjNum 747 snap 183 seq_num 747) 2023-03-29T14:08:51.294 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:51.297 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:51.297 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:51.392 INFO:tasks.rados.rados.0.smithi017.stdout:1550: done (2 left) 2023-03-29T14:08:51.392 INFO:tasks.rados.rados.0.smithi017.stdout:1552: done (1 left) 2023-03-29T14:08:51.392 INFO:tasks.rados.rados.0.smithi017.stdout:1556: done (0 left) 2023-03-29T14:08:51.392 INFO:tasks.rados.rados.0.smithi017.stdout:1557: write oid 47 current snap is 183 2023-03-29T14:08:51.392 INFO:tasks.rados.rados.0.smithi017.stdout:1557: seq_num 750 ranges {605250=639344,1855142=452579} 2023-03-29T14:08:51.399 INFO:tasks.rados.rados.0.smithi017.stdout:1557: writing smithi017107884-47 from 605250 to 1244594 tid 1 2023-03-29T14:08:51.405 INFO:tasks.rados.rados.0.smithi017.stdout:1557: writing smithi017107884-47 from 1855142 to 2307721 tid 2 2023-03-29T14:08:51.405 INFO:tasks.rados.rados.0.smithi017.stdout:1558: read oid 20 snap -1 2023-03-29T14:08:51.405 INFO:tasks.rados.rados.0.smithi017.stdout:1558: expect (ObjNum 699 snap 168 seq_num 699) 2023-03-29T14:08:51.405 INFO:tasks.rados.rados.0.smithi017.stdout:1559: snap_create 2023-03-29T14:08:51.525 INFO:tasks.rados.rados.0.smithi017.stdout:1557: finishing write tid 1 to smithi017107884-47 2023-03-29T14:08:51.552 INFO:tasks.rados.rados.0.smithi017.stdout:1558: expect (ObjNum 699 snap 168 seq_num 699) 2023-03-29T14:08:51.861 INFO:tasks.rados.rados.0.smithi017.stdout:1557: finishing write tid 2 to smithi017107884-47 2023-03-29T14:08:51.862 INFO:tasks.rados.rados.0.smithi017.stdout:1557: finishing write tid 3 to smithi017107884-47 2023-03-29T14:08:51.862 INFO:tasks.rados.rados.0.smithi017.stdout:1557: finishing write tid 4 to smithi017107884-47 2023-03-29T14:08:51.862 INFO:tasks.rados.rados.0.smithi017.stdout:1557: oid 47 updating version 0 to 225 2023-03-29T14:08:51.862 INFO:tasks.rados.rados.0.smithi017.stdout:1557: oid 47 version 225 is already newer than 225 2023-03-29T14:08:51.862 INFO:tasks.rados.rados.0.smithi017.stdout:1557: oid 47 version 225 is already newer than 225 2023-03-29T14:08:51.862 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 225 (ObjNum 750 snap 183 seq_num 750) dirty exists 2023-03-29T14:08:51.862 INFO:tasks.rados.rados.0.smithi017.stdout:1557: left oid 47 (ObjNum 750 snap 183 seq_num 750) 2023-03-29T14:08:52.284 INFO:tasks.rados.rados.0.smithi017.stdout:1557: done (2 left) 2023-03-29T14:08:52.284 INFO:tasks.rados.rados.0.smithi017.stdout:1558: done (1 left) 2023-03-29T14:08:52.284 INFO:tasks.rados.rados.0.smithi017.stdout:1559: done (0 left) 2023-03-29T14:08:52.284 INFO:tasks.rados.rados.0.smithi017.stdout:1560: write oid 6 current snap is 184 2023-03-29T14:08:52.284 INFO:tasks.rados.rados.0.smithi017.stdout:1560: seq_num 751 ranges {622057=714593,1913624=708433} 2023-03-29T14:08:52.291 INFO:tasks.rados.rados.0.smithi017.stdout:1560: writing smithi017107884-6 from 622057 to 1336650 tid 1 2023-03-29T14:08:52.298 INFO:tasks.rados.rados.0.smithi017.stdout:1560: writing smithi017107884-6 from 1913624 to 2622057 tid 2 2023-03-29T14:08:52.298 INFO:tasks.rados.rados.0.smithi017.stdout:1561: read oid 20 snap -1 2023-03-29T14:08:52.298 INFO:tasks.rados.rados.0.smithi017.stdout:1561: expect (ObjNum 699 snap 168 seq_num 699) 2023-03-29T14:08:52.298 INFO:tasks.rados.rados.0.smithi017.stdout:1562: delete oid 45 current snap is 184 2023-03-29T14:08:52.422 INFO:tasks.rados.rados.0.smithi017.stdout:1560: finishing write tid 1 to smithi017107884-6 2023-03-29T14:08:52.437 INFO:tasks.rados.rados.0.smithi017.stdout:1560: finishing write tid 2 to smithi017107884-6 2023-03-29T14:08:52.446 INFO:tasks.rados.rados.0.smithi017.stdout:1560: finishing write tid 3 to smithi017107884-6 2023-03-29T14:08:52.447 INFO:tasks.rados.rados.0.smithi017.stdout:1560: finishing write tid 4 to smithi017107884-6 2023-03-29T14:08:52.448 INFO:tasks.rados.rados.0.smithi017.stdout:1560: oid 6 updating version 0 to 184 2023-03-29T14:08:52.448 INFO:tasks.rados.rados.0.smithi017.stdout:1560: oid 6 version 184 is already newer than 184 2023-03-29T14:08:52.448 INFO:tasks.rados.rados.0.smithi017.stdout:1560: oid 6 version 184 is already newer than 184 2023-03-29T14:08:52.448 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 184 (ObjNum 751 snap 184 seq_num 751) dirty exists 2023-03-29T14:08:52.448 INFO:tasks.rados.rados.0.smithi017.stdout:1560: left oid 6 (ObjNum 751 snap 184 seq_num 751) 2023-03-29T14:08:52.448 INFO:tasks.rados.rados.0.smithi017.stdout:1561: expect (ObjNum 699 snap 168 seq_num 699) 2023-03-29T14:08:52.759 INFO:tasks.rados.rados.0.smithi017.stdout:1560: done (2 left) 2023-03-29T14:08:52.759 INFO:tasks.rados.rados.0.smithi017.stdout:1561: done (1 left) 2023-03-29T14:08:52.759 INFO:tasks.rados.rados.0.smithi017.stdout:1562: done (0 left) 2023-03-29T14:08:52.759 INFO:tasks.rados.rados.0.smithi017.stdout:1563: write (excl) oid 33 current snap is 184 2023-03-29T14:08:52.759 INFO:tasks.rados.rados.0.smithi017.stdout:1563: seq_num 753 ranges {655671=581444,2030588=625083} 2023-03-29T14:08:52.766 INFO:tasks.rados.rados.0.smithi017.stdout:1563: writing smithi017107884-33 from 655671 to 1237115 tid 1 2023-03-29T14:08:52.774 INFO:tasks.rados.rados.0.smithi017.stdout:1563: writing smithi017107884-33 from 2030588 to 2655671 tid 2 2023-03-29T14:08:52.774 INFO:tasks.rados.rados.0.smithi017.stdout:1564: snap_remove snap 180 2023-03-29T14:08:53.253 INFO:tasks.rados.rados.0.smithi017.stdout:1564: done (1 left) 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1565: write oid 27 current snap is 184 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1563: finishing write tid 1 to smithi017107884-33 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1563: finishing write tid 2 to smithi017107884-33 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1563: finishing write tid 3 to smithi017107884-33 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1563: finishing write tid 4 to smithi017107884-33 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1563: oid 33 updating version 0 to 213 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1563: oid 33 updating version 213 to 215 2023-03-29T14:08:53.254 INFO:tasks.rados.rados.0.smithi017.stdout:1563: oid 33 version 215 is already newer than 215 2023-03-29T14:08:53.255 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 215 (ObjNum 753 snap 184 seq_num 753) dirty exists 2023-03-29T14:08:53.255 INFO:tasks.rados.rados.0.smithi017.stdout:1563: left oid 33 (ObjNum 753 snap 184 seq_num 753) 2023-03-29T14:08:53.255 INFO:tasks.rados.rados.0.smithi017.stdout:1565: seq_num 754 ranges {672478=656693,2089070=459917} 2023-03-29T14:08:53.260 INFO:tasks.rados.rados.0.smithi017.stdout:1565: writing smithi017107884-27 from 672478 to 1329171 tid 1 2023-03-29T14:08:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1565: writing smithi017107884-27 from 2089070 to 2548987 tid 2 2023-03-29T14:08:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1563: done (1 left) 2023-03-29T14:08:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1566: write (excl) oid 16 current snap is 184 2023-03-29T14:08:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1566: seq_num 755 ranges {689285=731942,2147552=541733} 2023-03-29T14:08:53.275 INFO:tasks.rados.rados.0.smithi017.stdout:1566: writing smithi017107884-16 from 689285 to 1421227 tid 1 2023-03-29T14:08:53.282 INFO:tasks.rados.rados.0.smithi017.stdout:1566: writing smithi017107884-16 from 2147552 to 2689285 tid 2 2023-03-29T14:08:53.282 INFO:tasks.rados.rados.0.smithi017.stdout:1567: snap_remove snap 178 2023-03-29T14:08:53.299 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:53.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:53.430+0000 7f025ccab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:53.434+0000 7f025ccab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:53.497 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:53.500 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:53.500 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:54.288 INFO:tasks.rados.rados.0.smithi017.stdout:1567: done (2 left) 2023-03-29T14:08:54.288 INFO:tasks.rados.rados.0.smithi017.stdout:1568: read oid 3 snap -1 2023-03-29T14:08:54.288 INFO:tasks.rados.rados.0.smithi017.stdout:1568: expect (ObjNum 747 snap 183 seq_num 747) 2023-03-29T14:08:54.288 INFO:tasks.rados.rados.0.smithi017.stdout:1569: write (excl) oid 40 current snap is 184 2023-03-29T14:08:54.288 INFO:tasks.rados.rados.0.smithi017.stdout:1566: finishing write tid 1 to smithi017107884-16 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: finishing write tid 1 to smithi017107884-27 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: finishing write tid 2 to smithi017107884-27 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: finishing write tid 3 to smithi017107884-27 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: finishing write tid 4 to smithi017107884-27 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: oid 27 updating version 0 to 188 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: oid 27 version 188 is already newer than 188 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: oid 27 version 188 is already newer than 188 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 188 (ObjNum 754 snap 184 seq_num 754) dirty exists 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1565: left oid 27 (ObjNum 754 snap 184 seq_num 754) 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1566: finishing write tid 2 to smithi017107884-16 2023-03-29T14:08:54.289 INFO:tasks.rados.rados.0.smithi017.stdout:1569: seq_num 756 ranges {706092=407191,1806034=463586} 2023-03-29T14:08:54.294 INFO:tasks.rados.rados.0.smithi017.stdout:1569: writing smithi017107884-40 from 706092 to 1113283 tid 1 2023-03-29T14:08:54.300 INFO:tasks.rados.rados.0.smithi017.stdout:1569: writing smithi017107884-40 from 1806034 to 2269620 tid 2 2023-03-29T14:08:54.300 INFO:tasks.rados.rados.0.smithi017.stdout:1565: done (3 left) 2023-03-29T14:08:54.301 INFO:tasks.rados.rados.0.smithi017.stdout:1570: snap_remove snap 183 2023-03-29T14:08:54.871 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:08:54.871 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:08:54.871 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:08:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:54.999+0000 7f5015ab9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:55.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:55.005+0000 7f5015ab9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:08:55.209 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:08:55.221 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2023-03-29T14:08:54.286549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":228994628,"num_objects":852,"num_object_clones":756,"num_object_copies":2556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":3469,"num_read_kb":2024525,"num_write":4792,"num_write_kb":1315260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2526,"ondisk_log_size":2526,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1666184,"kb_used_data":1585512,"kb_used_omap":0,"kb_used_meta":80640,"kb_avail":375821176,"statfs":{"total":386547056640,"available":384840884224,"internally_reserved":0,"allocated":1623564288,"data_stored":1992548207,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":82575360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7294040,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":18876,"num_write":24,"num_write_kb":6638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.011370"},"pg_stats":[{"pgid":"3.f","version":"373'116","reported_seq":359,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.453442+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:49.453442+0000","last_peered":"2023-03-29T14:08:49.453442+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:49.453442+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12336932,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":65943,"num_write":146,"num_write_kb":42272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"361'44","reported_seq":389,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.362518+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:49.362518+0000","last_peered":"2023-03-29T14:08:49.362518+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:49.362518+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7563414,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":25432,"num_write":82,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"380'241","reported_seq":863,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-03-29T14:08:49.458601+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:49.458601+0000","last_peered":"2023-03-29T14:08:49.458601+0000","last_clean":"2023-03-29T14:08:49.458601+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:49.458601+0000","last_undegraded":"2023-03-29T14:08:49.458601+0000","last_fullsized":"2023-03-29T14:08:49.458601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31127950,"num_objects":86,"num_object_clones":76,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":251013,"num_write":466,"num_write_kb":138376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"379'239","reported_seq":852,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-03-29T14:08:49.457835+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:49.457835+0000","last_peered":"2023-03-29T14:08:49.457835+0000","last_clean":"2023-03-29T14:08:49.457835+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:49.457835+0000","last_undegraded":"2023-03-29T14:08:49.457835+0000","last_fullsized":"2023-03-29T14:08:49.457835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245302,"num_objects":94,"num_object_clones":86,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":337,"num_read_kb":206402,"num_write":468,"num_write_kb":133708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"371'204","reported_seq":748,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-03-29T14:08:49.312334+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:49.312334+0000","last_peered":"2023-03-29T14:08:49.312334+0000","last_clean":"2023-03-29T14:08:49.312334+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:49.312334+0000","last_undegraded":"2023-03-29T14:08:49.312334+0000","last_fullsized":"2023-03-29T14:08:49.312334+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21850288,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":270,"num_read_kb":160561,"num_write":408,"num_write_kb":119746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":323,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.455778+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:49.455778+0000","last_peered":"2023-03-29T14:08:49.455778+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:49.455778+0000","last_undegraded":"2023-03-29T14:08:49.455778+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"377'135","reported_seq":603,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.449239+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:49.449239+0000","last_peered":"2023-03-29T14:08:49.449239+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:49.449239+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9681492,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":149,"num_read_kb":67879,"num_write":246,"num_write_kb":69400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":48}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":498,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-03-29T14:08:49.465296+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:49.465296+0000","last_peered":"2023-03-29T14:08:49.465296+0000","last_clean":"2023-03-29T14:08:49.465296+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:49.465296+0000","last_undegraded":"2023-03-29T14:08:49.465296+0000","last_fullsized":"2023-03-29T14:08:49.465296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":111,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":300,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.360428+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:49.360428+0000","last_peered":"2023-03-29T14:08:49.360428+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:49.360428+0000","last_undegraded":"2023-03-29T14:08:49.360428+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"374'94","reported_seq":279,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.337917+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:49.337917+0000","last_peered":"2023-03-29T14:08:49.337917+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:49.337917+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9244510,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":12595,"num_write":112,"num_write_kb":29186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":326,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.447961+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:49.447961+0000","last_peered":"2023-03-29T14:08:49.447961+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:49.447961+0000","last_undegraded":"2023-03-29T14:08:49.447961+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"382'95","reported_seq":525,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.367861+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:49.367861+0000","last_peered":"2023-03-29T14:08:49.367861+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:49.367861+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":158,"num_read_kb":107602,"num_write":182,"num_write_kb":52894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.333954+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:49.333954+0000","last_peered":"2023-03-29T14:08:49.333954+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:49.333954+0000","last_undegraded":"2023-03-29T14:08:49.333954+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"368'131","reported_seq":603,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.439553+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:49.439553+0000","last_peered":"2023-03-29T14:08:49.439553+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:49.439553+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10925078,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":216,"num_read_kb":130736,"num_write":252,"num_write_kb":74172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":326,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.442124+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:49.442124+0000","last_peered":"2023-03-29T14:08:49.442124+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:49.442124+0000","last_undegraded":"2023-03-29T14:08:49.442124+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":525,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.338681+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:49.338681+0000","last_peered":"2023-03-29T14:08:49.338681+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:49.338681+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"373'180","reported_seq":771,"reported_epoch":383,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.454237+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:49.454237+0000","last_peered":"2023-03-29T14:08:49.454237+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:49.454237+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043222,"num_objects":64,"num_object_clones":56,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":339,"num_read_kb":232829,"num_write":350,"num_write_kb":101254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":64}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":326,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.451597+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:49.451597+0000","last_peered":"2023-03-29T14:08:49.451597+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:49.451597+0000","last_undegraded":"2023-03-29T14:08:49.451597+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"382'325","reported_seq":1039,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.543387+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:49.543387+0000","last_peered":"2023-03-29T14:08:49.543387+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:49.543387+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31498166,"num_objects":114,"num_object_clones":100,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":490,"num_read_kb":296488,"num_write":612,"num_write_kb":176924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":114}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":300,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.345548+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:49.345548+0000","last_peered":"2023-03-29T14:08:49.345548+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:49.345548+0000","last_undegraded":"2023-03-29T14:08:49.345548+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"372'116","reported_seq":546,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-03-29T14:08:49.336381+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:49.336381+0000","last_peered":"2023-03-29T14:08:49.336381+0000","last_clean":"2023-03-29T14:08:49.336381+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:49.336381+0000","last_undegraded":"2023-03-29T14:08:49.336381+0000","last_fullsized":"2023-03-29T14:08:49.336381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":42,"num_object_clones":38,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":79797,"num_write":234,"num_write_kb":68058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"376'220","reported_seq":764,"reported_epoch":383,"state":"active+clean","last_fresh":"2023-03-29T14:08:49.343249+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:49.343249+0000","last_peered":"2023-03-29T14:08:49.343249+0000","last_clean":"2023-03-29T14:08:49.343249+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:49.343249+0000","last_undegraded":"2023-03-29T14:08:49.343249+0000","last_fullsized":"2023-03-29T14:08:49.343249+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16774310,"num_objects":78,"num_object_clones":68,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":271,"num_read_kb":159209,"num_write":404,"num_write_kb":117416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":300,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.317985+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:49.317985+0000","last_peered":"2023-03-29T14:08:49.317985+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:49.317985+0000","last_undegraded":"2023-03-29T14:08:49.317985+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":326,"reported_epoch":383,"state":"active+undersized","last_fresh":"2023-03-29T14:08:49.436752+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:49.436752+0000","last_peered":"2023-03-29T14:08:49.436752+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:49.436752+0000","last_undegraded":"2023-03-29T14:08:49.436752+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"380'210","reported_seq":769,"reported_epoch":384,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:49.343199+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:49.343199+0000","last_peered":"2023-03-29T14:08:49.343199+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:49.343199+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23967904,"num_objects":74,"num_object_clones":66,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":285,"num_read_kb":169952,"num_write":404,"num_write_kb":119342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":74}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":227813892,"num_objects":848,"num_object_clones":756,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":446,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":20,"num_read":3327,"num_read_kb":2024405,"num_write":4518,"num_write_kb":1311472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2429,"ondisk_log_size":2429,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":486760,"kb_used_data":464416,"kb_used_omap":0,"kb_used_meta":22336,"kb_avail":93885080,"statfs":{"total":96636764160,"available":96138321920,"internally_reserved":0,"allocated":475561984,"data_stored":585986133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22872064},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":538332,"kb_used_data":514772,"kb_used_omap":0,"kb_used_meta":23552,"kb_avail":93833508,"statfs":{"total":96636764160,"available":96085512192,"internally_reserved":0,"allocated":527126528,"data_stored":650217898,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24117248},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":485148,"kb_used_data":462484,"kb_used_omap":0,"kb_used_meta":22656,"kb_avail":93886692,"statfs":{"total":96636764160,"available":96139972608,"internally_reserved":0,"allocated":473583616,"data_stored":583111951,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23199744},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:08:55.291 INFO:tasks.rados.rados.0.smithi017.stdout:1570: done (3 left) 2023-03-29T14:08:55.291 INFO:tasks.rados.rados.0.smithi017.stdout:1571: snap_create 2023-03-29T14:08:55.291 INFO:tasks.rados.rados.0.smithi017.stdout:1566: finishing write tid 3 to smithi017107884-16 2023-03-29T14:08:55.291 INFO:tasks.rados.rados.0.smithi017.stdout:1566: finishing write tid 4 to smithi017107884-16 2023-03-29T14:08:55.291 INFO:tasks.rados.rados.0.smithi017.stdout:1566: oid 16 updating version 0 to 117 2023-03-29T14:08:55.292 INFO:tasks.rados.rados.0.smithi017.stdout:1566: oid 16 updating version 117 to 120 2023-03-29T14:08:55.292 INFO:tasks.rados.rados.0.smithi017.stdout:1566: oid 16 version 120 is already newer than 120 2023-03-29T14:08:55.292 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 120 (ObjNum 755 snap 184 seq_num 755) dirty exists 2023-03-29T14:08:55.292 INFO:tasks.rados.rados.0.smithi017.stdout:1566: left oid 16 (ObjNum 755 snap 184 seq_num 755) 2023-03-29T14:08:55.292 INFO:tasks.rados.rados.0.smithi017.stdout:1568: expect (ObjNum 747 snap 183 seq_num 747) 2023-03-29T14:08:55.433 INFO:tasks.rados.rados.0.smithi017.stdout:1569: finishing write tid 1 to smithi017107884-40 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:1569: finishing write tid 2 to smithi017107884-40 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:1569: finishing write tid 3 to smithi017107884-40 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:1569: finishing write tid 4 to smithi017107884-40 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:1569: oid 40 updating version 0 to 243 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:1569: oid 40 version 243 is already newer than 243 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:1569: oid 40 version 243 is already newer than 240 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 243 (ObjNum 756 snap 184 seq_num 756) dirty exists 2023-03-29T14:08:55.434 INFO:tasks.rados.rados.0.smithi017.stdout:1569: left oid 40 (ObjNum 756 snap 184 seq_num 756) 2023-03-29T14:08:55.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:08:55.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:55.630+0000 7f2474981700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:55.634 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:55.634+0000 7f2474981700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:55.696 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:55.699 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:55.699 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:55.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~226s 2023-03-29T14:08:56.295 INFO:tasks.rados.rados.0.smithi017.stdout:1566: done (3 left) 2023-03-29T14:08:56.295 INFO:tasks.rados.rados.0.smithi017.stdout:1568: done (2 left) 2023-03-29T14:08:56.295 INFO:tasks.rados.rados.0.smithi017.stdout:1569: done (1 left) 2023-03-29T14:08:56.296 INFO:tasks.rados.rados.0.smithi017.stdout:1571: done (0 left) 2023-03-29T14:08:56.296 INFO:tasks.rados.rados.0.smithi017.stdout:1572: read oid 11 snap 181 2023-03-29T14:08:56.296 INFO:tasks.rados.rados.0.smithi017.stdout:1572: expect (ObjNum 674 snap 164 seq_num 674) 2023-03-29T14:08:56.296 INFO:tasks.rados.rados.0.smithi017.stdout:1573: snap_create 2023-03-29T14:08:56.404 INFO:tasks.rados.rados.0.smithi017.stdout:1572: expect (ObjNum 674 snap 164 seq_num 674) 2023-03-29T14:08:57.298 INFO:tasks.rados.rados.0.smithi017.stdout:1572: done (1 left) 2023-03-29T14:08:57.298 INFO:tasks.rados.rados.0.smithi017.stdout:1573: done (0 left) 2023-03-29T14:08:57.298 INFO:tasks.rados.rados.0.smithi017.stdout:1574: write oid 50 current snap is 186 2023-03-29T14:08:57.299 INFO:tasks.rados.rados.0.smithi017.stdout:1574: seq_num 757 ranges {722899=482440,1748163=723597} 2023-03-29T14:08:57.305 INFO:tasks.rados.rados.0.smithi017.stdout:1574: writing smithi017107884-50 from 722899 to 1205339 tid 1 2023-03-29T14:08:57.312 INFO:tasks.rados.rados.0.smithi017.stdout:1574: writing smithi017107884-50 from 1748163 to 2471760 tid 2 2023-03-29T14:08:57.312 INFO:tasks.rados.rados.0.smithi017.stdout:1575: delete oid 28 current snap is 186 2023-03-29T14:08:57.343 INFO:tasks.rados.rados.0.smithi017.stdout:1575: done (1 left) 2023-03-29T14:08:57.344 INFO:tasks.rados.rados.0.smithi017.stdout:1576: delete oid 27 current snap is 186 2023-03-29T14:08:57.543 INFO:tasks.rados.rados.0.smithi017.stdout:1576: done (1 left) 2023-03-29T14:08:57.543 INFO:tasks.rados.rados.0.smithi017.stdout:1577: delete oid 33 current snap is 186 2023-03-29T14:08:57.570 INFO:tasks.rados.rados.0.smithi017.stdout:1574: finishing write tid 1 to smithi017107884-50 2023-03-29T14:08:57.594 INFO:tasks.rados.rados.0.smithi017.stdout:1574: finishing write tid 2 to smithi017107884-50 2023-03-29T14:08:57.601 INFO:tasks.rados.rados.0.smithi017.stdout:1577: done (1 left) 2023-03-29T14:08:57.601 INFO:tasks.rados.rados.0.smithi017.stdout:1578: read oid 14 snap -1 2023-03-29T14:08:57.601 INFO:tasks.rados.rados.0.smithi017.stdout:1578: expect (ObjNum 740 snap 179 seq_num 740) 2023-03-29T14:08:57.601 INFO:tasks.rados.rados.0.smithi017.stdout:1579: read oid 23 snap -1 2023-03-29T14:08:57.601 INFO:tasks.rados.rados.0.smithi017.stdout:1579: expect (ObjNum 719 snap 177 seq_num 719) 2023-03-29T14:08:57.601 INFO:tasks.rados.rados.0.smithi017.stdout:1580: write (excl) oid 37 current snap is 186 2023-03-29T14:08:57.601 INFO:tasks.rados.rados.0.smithi017.stdout:1580: seq_num 761 ranges {790127=499789,1698444=730935} 2023-03-29T14:08:57.608 INFO:tasks.rados.rados.0.smithi017.stdout:1580: writing smithi017107884-37 from 790127 to 1289916 tid 1 2023-03-29T14:08:57.616 INFO:tasks.rados.rados.0.smithi017.stdout:1580: writing smithi017107884-37 from 1698444 to 2429379 tid 2 2023-03-29T14:08:57.616 INFO:tasks.rados.rados.0.smithi017.stdout:1581: snap_create 2023-03-29T14:08:57.616 INFO:tasks.rados.rados.0.smithi017.stdout:1578: expect (ObjNum 740 snap 179 seq_num 740) 2023-03-29T14:08:57.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:08:57.744 INFO:tasks.rados.rados.0.smithi017.stdout:1574: finishing write tid 3 to smithi017107884-50 2023-03-29T14:08:57.745 INFO:tasks.rados.rados.0.smithi017.stdout:1579: expect (ObjNum 719 snap 177 seq_num 719) 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1574: finishing write tid 4 to smithi017107884-50 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1574: oid 50 updating version 0 to 135 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1574: oid 50 version 135 is already newer than 135 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1574: oid 50 version 135 is already newer than 135 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 135 (ObjNum 757 snap 186 seq_num 757) dirty exists 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1574: left oid 50 (ObjNum 757 snap 186 seq_num 757) 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1580: finishing write tid 1 to smithi017107884-37 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1580: finishing write tid 2 to smithi017107884-37 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1580: finishing write tid 3 to smithi017107884-37 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1580: finishing write tid 4 to smithi017107884-37 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1580: oid 37 updating version 0 to 98 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1580: oid 37 version 98 is already newer than 95 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:1580: oid 37 version 98 is already newer than 98 2023-03-29T14:08:57.826 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 98 (ObjNum 761 snap 186 seq_num 761) dirty exists 2023-03-29T14:08:57.827 INFO:tasks.rados.rados.0.smithi017.stdout:1580: left oid 37 (ObjNum 761 snap 186 seq_num 761) 2023-03-29T14:08:57.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:57.829+0000 7fe78d3de700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:57.833 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:08:57.833+0000 7fe78d3de700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:08:57.898 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:08:57.900 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:08:57.900 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:08:58.302 INFO:tasks.rados.rados.0.smithi017.stdout:1574: done (4 left) 2023-03-29T14:08:58.302 INFO:tasks.rados.rados.0.smithi017.stdout:1578: done (3 left) 2023-03-29T14:08:58.302 INFO:tasks.rados.rados.0.smithi017.stdout:1579: done (2 left) 2023-03-29T14:08:58.302 INFO:tasks.rados.rados.0.smithi017.stdout:1580: done (1 left) 2023-03-29T14:08:58.302 INFO:tasks.rados.rados.0.smithi017.stdout:1581: done (0 left) 2023-03-29T14:08:58.302 INFO:tasks.rados.rados.0.smithi017.stdout:1582: write (excl) oid 18 current snap is 187 2023-03-29T14:08:58.303 INFO:tasks.rados.rados.0.smithi017.stdout:1582: seq_num 762 ranges {406934=575038,1640573=474593,2642791=164143} 2023-03-29T14:08:58.307 INFO:tasks.rados.rados.0.smithi017.stdout:1582: writing smithi017107884-18 from 406934 to 981972 tid 1 2023-03-29T14:08:58.313 INFO:tasks.rados.rados.0.smithi017.stdout:1582: writing smithi017107884-18 from 1640573 to 2115166 tid 2 2023-03-29T14:08:58.317 INFO:tasks.rados.rados.0.smithi017.stdout:1582: writing smithi017107884-18 from 2642791 to 2806934 tid 3 2023-03-29T14:08:58.317 INFO:tasks.rados.rados.0.smithi017.stdout:1583: snap_create 2023-03-29T14:08:58.379 INFO:tasks.rados.rados.0.smithi017.stdout:1582: finishing write tid 1 to smithi017107884-18 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: finishing write tid 2 to smithi017107884-18 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: finishing write tid 3 to smithi017107884-18 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: finishing write tid 4 to smithi017107884-18 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: finishing write tid 5 to smithi017107884-18 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: oid 18 updating version 0 to 136 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: oid 18 updating version 136 to 139 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: oid 18 version 139 is already newer than 139 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: oid 18 version 139 is already newer than 139 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 139 (ObjNum 762 snap 187 seq_num 762) dirty exists 2023-03-29T14:08:58.440 INFO:tasks.rados.rados.0.smithi017.stdout:1582: left oid 18 (ObjNum 762 snap 187 seq_num 762) 2023-03-29T14:08:59.303 INFO:tasks.rados.rados.0.smithi017.stdout:1582: done (1 left) 2023-03-29T14:08:59.303 INFO:tasks.rados.rados.0.smithi017.stdout:1583: done (0 left) 2023-03-29T14:08:59.303 INFO:tasks.rados.rados.0.smithi017.stdout:1584: snap_remove snap 184 2023-03-29T14:08:59.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:00.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:00.029+0000 7fa89c123700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:00.033+0000 7fa89c123700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:00.096 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:00.098 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:00.099 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:00.225 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:00.225 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:00.225 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:00.310 INFO:tasks.rados.rados.0.smithi017.stdout:1584: done (0 left) 2023-03-29T14:09:00.310 INFO:tasks.rados.rados.0.smithi017.stdout:1585: delete oid 28 current snap is 188 2023-03-29T14:09:00.349 INFO:tasks.rados.rados.0.smithi017.stdout:1585: done (0 left) 2023-03-29T14:09:00.350 INFO:tasks.rados.rados.0.smithi017.stdout:1586: write oid 17 current snap is 188 2023-03-29T14:09:00.350 INFO:tasks.rados.rados.0.smithi017.stdout:1586: seq_num 764 ranges {440548=725536,1757537=478262,2813990=26558} 2023-03-29T14:09:00.356 INFO:tasks.rados.rados.0.smithi017.stdout:1586: writing smithi017107884-17 from 440548 to 1166084 tid 1 2023-03-29T14:09:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:00.353+0000 7f2a82963700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:00.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:00.360+0000 7f2a82963700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:00.363 INFO:tasks.rados.rados.0.smithi017.stdout:1586: writing smithi017107884-17 from 1757537 to 2235799 tid 2 2023-03-29T14:09:00.367 INFO:tasks.rados.rados.0.smithi017.stdout:1586: writing smithi017107884-17 from 2813990 to 2840548 tid 3 2023-03-29T14:09:00.367 INFO:tasks.rados.rados.0.smithi017.stdout:1587: write (excl) oid 9 current snap is 188 2023-03-29T14:09:00.367 INFO:tasks.rados.rados.0.smithi017.stdout:1587: seq_num 765 ranges {457355=400785,1416019=621920,2725060=132295} 2023-03-29T14:09:00.372 INFO:tasks.rados.rados.0.smithi017.stdout:1587: writing smithi017107884-9 from 457355 to 858140 tid 1 2023-03-29T14:09:00.379 INFO:tasks.rados.rados.0.smithi017.stdout:1587: writing smithi017107884-9 from 1416019 to 2037939 tid 2 2023-03-29T14:09:00.383 INFO:tasks.rados.rados.0.smithi017.stdout:1587: writing smithi017107884-9 from 2725060 to 2857355 tid 3 2023-03-29T14:09:00.384 INFO:tasks.rados.rados.0.smithi017.stdout:1588: read oid 6 snap -1 2023-03-29T14:09:00.384 INFO:tasks.rados.rados.0.smithi017.stdout:1588: expect (ObjNum 751 snap 184 seq_num 751) 2023-03-29T14:09:00.384 INFO:tasks.rados.rados.0.smithi017.stdout:1589: write oid 40 current snap is 188 2023-03-29T14:09:00.384 INFO:tasks.rados.rados.0.smithi017.stdout:1589: seq_num 766 ranges {474162=476034,1358148=481931,2352483=465303} 2023-03-29T14:09:00.389 INFO:tasks.rados.rados.0.smithi017.stdout:1589: writing smithi017107884-40 from 474162 to 950196 tid 1 2023-03-29T14:09:00.394 INFO:tasks.rados.rados.0.smithi017.stdout:1589: writing smithi017107884-40 from 1358148 to 1840079 tid 2 2023-03-29T14:09:00.399 INFO:tasks.rados.rados.0.smithi017.stdout:1589: writing smithi017107884-40 from 2352483 to 2817786 tid 3 2023-03-29T14:09:00.399 INFO:tasks.rados.rados.0.smithi017.stdout:1590: write oid 45 current snap is 188 2023-03-29T14:09:00.399 INFO:tasks.rados.rados.0.smithi017.stdout:1590: seq_num 767 ranges {490969=551283,1816630=625589} 2023-03-29T14:09:00.405 INFO:tasks.rados.rados.0.smithi017.stdout:1590: writing smithi017107884-45 from 490969 to 1042252 tid 1 2023-03-29T14:09:00.413 INFO:tasks.rados.rados.0.smithi017.stdout:1590: writing smithi017107884-45 from 1816630 to 2442219 tid 2 2023-03-29T14:09:00.413 INFO:tasks.rados.rados.0.smithi017.stdout:1591: delete oid 18 current snap is 188 2023-03-29T14:09:00.436 INFO:tasks.rados.rados.0.smithi017.stdout:1588: expect (ObjNum 751 snap 184 seq_num 751) 2023-03-29T14:09:00.589 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:00.591 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: finishing write tid 1 to smithi017107884-17 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: finishing write tid 2 to smithi017107884-17 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1588: done (5 left) 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1591: done (4 left) 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1592: write (excl) oid 42 current snap is 188 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: finishing write tid 3 to smithi017107884-17 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: finishing write tid 4 to smithi017107884-17 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: finishing write tid 5 to smithi017107884-17 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: oid 17 updating version 0 to 246 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: oid 17 version 246 is already newer than 246 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: oid 17 version 246 is already newer than 246 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: oid 17 version 246 is already newer than 246 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 17 v 246 (ObjNum 764 snap 188 seq_num 764) dirty exists 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1586: left oid 17 (ObjNum 764 snap 188 seq_num 764) 2023-03-29T14:09:00.596 INFO:tasks.rados.rados.0.smithi017.stdout:1587: finishing write tid 1 to smithi017107884-9 2023-03-29T14:09:00.597 INFO:tasks.rados.rados.0.smithi017.stdout:1590: finishing write tid 1 to smithi017107884-45 2023-03-29T14:09:00.597 INFO:tasks.rados.rados.0.smithi017.stdout:1590: finishing write tid 2 to smithi017107884-45 2023-03-29T14:09:00.597 INFO:tasks.rados.rados.0.smithi017.stdout:1590: finishing write tid 3 to smithi017107884-45 2023-03-29T14:09:00.597 INFO:tasks.rados.rados.0.smithi017.stdout:1592: seq_num 769 ranges {524583=418134,1649947=629258,2834752=89831} 2023-03-29T14:09:00.603 INFO:tasks.rados.rados.0.smithi017.stdout:1592: writing smithi017107884-42 from 524583 to 942717 tid 1 2023-03-29T14:09:00.606 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":594,"stamp":"2023-03-29T14:09:00.308147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222036398,"num_objects":866,"num_object_clones":770,"num_object_copies":2598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3506,"num_read_kb":2050385,"num_write":4840,"num_write_kb":1328908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2552,"ondisk_log_size":2552,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1682344,"kb_used_data":1601288,"kb_used_omap":0,"kb_used_meta":81024,"kb_avail":375805016,"statfs":{"total":386547056640,"available":384824336384,"internally_reserved":0,"allocated":1639718912,"data_stored":2014002990,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":82968576},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5983490,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":25,"num_read_kb":19679,"num_write":40,"num_write_kb":11290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014464"},"pg_stats":[{"pgid":"3.f","version":"386'120","reported_seq":371,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:55.304040+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:08:55.304040+0000","last_peered":"2023-03-29T14:08:55.304040+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:08:55.304040+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13076440,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":65944,"num_write":154,"num_write_kb":44762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":24}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"361'44","reported_seq":391,"reported_epoch":385,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:51.478399+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:08:51.478399+0000","last_peered":"2023-03-29T14:08:51.478399+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:08:51.478399+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7563414,"num_objects":18,"num_object_clones":16,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":25432,"num_write":82,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":18}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"380'241","reported_seq":868,"reported_epoch":389,"state":"active+clean","last_fresh":"2023-03-29T14:08:55.313217+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:08:55.313217+0000","last_peered":"2023-03-29T14:08:55.313217+0000","last_clean":"2023-03-29T14:08:55.313217+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:08:55.313217+0000","last_undegraded":"2023-03-29T14:08:55.313217+0000","last_fullsized":"2023-03-29T14:08:55.313217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31127950,"num_objects":86,"num_object_clones":76,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":251013,"num_write":466,"num_write_kb":138376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"387'243","reported_seq":864,"reported_epoch":389,"state":"active+clean","last_fresh":"2023-03-29T14:08:55.301670+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:08:55.301670+0000","last_peered":"2023-03-29T14:08:55.301670+0000","last_clean":"2023-03-29T14:08:55.301670+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:08:55.301670+0000","last_undegraded":"2023-03-29T14:08:55.301670+0000","last_fullsized":"2023-03-29T14:08:55.301670+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19631896,"num_objects":96,"num_object_clones":88,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":340,"num_read_kb":206403,"num_write":476,"num_write_kb":135410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"371'204","reported_seq":758,"reported_epoch":385,"state":"active+clean","last_fresh":"2023-03-29T14:08:52.445702+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:08:52.445702+0000","last_peered":"2023-03-29T14:08:52.445702+0000","last_clean":"2023-03-29T14:08:52.445702+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:08:52.445702+0000","last_undegraded":"2023-03-29T14:08:52.445702+0000","last_fullsized":"2023-03-29T14:08:52.445702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17816278,"num_objects":74,"num_object_clones":66,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":286,"num_read_kb":180235,"num_write":408,"num_write_kb":119746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":328,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-03-29T14:08:55.312474+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:08:55.312474+0000","last_peered":"2023-03-29T14:08:55.312474+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:08:55.312474+0000","last_undegraded":"2023-03-29T14:08:55.312474+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"377'135","reported_seq":608,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:55.302752+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:08:55.302752+0000","last_peered":"2023-03-29T14:08:55.302752+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:08:55.302752+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9681492,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":149,"num_read_kb":67879,"num_write":246,"num_write_kb":69400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":48}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"315'79","reported_seq":503,"reported_epoch":389,"state":"active+clean","last_fresh":"2023-03-29T14:08:55.313823+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:08:55.313823+0000","last_peered":"2023-03-29T14:08:55.313823+0000","last_clean":"2023-03-29T14:08:55.313823+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:08:55.313823+0000","last_undegraded":"2023-03-29T14:08:55.313823+0000","last_fullsized":"2023-03-29T14:08:55.313823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7731418,"num_objects":30,"num_object_clones":26,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":111,"num_read_kb":57967,"num_write":152,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":302,"reported_epoch":385,"state":"active+undersized","last_fresh":"2023-03-29T14:08:51.429472+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:51.429472+0000","last_peered":"2023-03-29T14:08:51.429472+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:51.429472+0000","last_undegraded":"2023-03-29T14:08:51.429472+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"374'94","reported_seq":282,"reported_epoch":387,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:52.824216+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:08:52.824216+0000","last_peered":"2023-03-29T14:08:52.824216+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:08:52.824216+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9244510,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":12595,"num_write":112,"num_write_kb":29186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":18}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":331,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-03-29T14:08:55.297300+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:08:55.297300+0000","last_peered":"2023-03-29T14:08:55.297300+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:08:55.297300+0000","last_undegraded":"2023-03-29T14:08:55.297300+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"382'95","reported_seq":528,"reported_epoch":387,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:52.792557+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:52.792557+0000","last_peered":"2023-03-29T14:08:52.792557+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:52.792557+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":158,"num_read_kb":107602,"num_write":182,"num_write_kb":52894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":385,"state":"active+undersized","last_fresh":"2023-03-29T14:08:51.448374+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:51.448374+0000","last_peered":"2023-03-29T14:08:51.448374+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:51.448374+0000","last_undegraded":"2023-03-29T14:08:51.448374+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"368'131","reported_seq":605,"reported_epoch":385,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:51.438393+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:08:51.438393+0000","last_peered":"2023-03-29T14:08:51.438393+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:08:51.438393+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10925078,"num_objects":48,"num_object_clones":42,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":216,"num_read_kb":130736,"num_write":252,"num_write_kb":74172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":331,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-03-29T14:08:55.305943+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:08:55.305943+0000","last_peered":"2023-03-29T14:08:55.305943+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:08:55.305943+0000","last_undegraded":"2023-03-29T14:08:55.305943+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":527,"reported_epoch":385,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:51.460305+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:51.460305+0000","last_peered":"2023-03-29T14:08:51.460305+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:51.460305+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"386'188","reported_seq":790,"reported_epoch":389,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:55.300938+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:08:55.300938+0000","last_peered":"2023-03-29T14:08:55.300938+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:08:55.300938+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18522238,"num_objects":68,"num_object_clones":60,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":341,"num_read_kb":232831,"num_write":366,"num_write_kb":106218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":68}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":331,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-03-29T14:08:55.314665+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:08:55.314665+0000","last_peered":"2023-03-29T14:08:55.314665+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:08:55.314665+0000","last_undegraded":"2023-03-29T14:08:55.314665+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"382'325","reported_seq":1045,"reported_epoch":387,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:54.404239+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:54.404239+0000","last_peered":"2023-03-29T14:08:54.404239+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:54.404239+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31498166,"num_objects":114,"num_object_clones":100,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":497,"num_read_kb":302668,"num_write":612,"num_write_kb":176924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":114}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":302,"reported_epoch":385,"state":"active+undersized","last_fresh":"2023-03-29T14:08:51.476582+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:51.476582+0000","last_peered":"2023-03-29T14:08:51.476582+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:51.476582+0000","last_undegraded":"2023-03-29T14:08:51.476582+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"372'116","reported_seq":548,"reported_epoch":385,"state":"active+clean","last_fresh":"2023-03-29T14:08:51.433496+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:51.433496+0000","last_peered":"2023-03-29T14:08:51.433496+0000","last_clean":"2023-03-29T14:08:51.433496+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:51.433496+0000","last_undegraded":"2023-03-29T14:08:51.433496+0000","last_fullsized":"2023-03-29T14:08:51.433496+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":42,"num_object_clones":38,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":79797,"num_write":234,"num_write_kb":68058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"384'225","reported_seq":773,"reported_epoch":385,"state":"active+clean","last_fresh":"2023-03-29T14:08:51.582154+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:51.582154+0000","last_peered":"2023-03-29T14:08:51.582154+0000","last_clean":"2023-03-29T14:08:51.582154+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:51.582154+0000","last_undegraded":"2023-03-29T14:08:51.582154+0000","last_fullsized":"2023-03-29T14:08:51.582154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14219712,"num_objects":80,"num_object_clones":70,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":272,"num_read_kb":159210,"num_write":412,"num_write_kb":119550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":385,"state":"active+undersized","last_fresh":"2023-03-29T14:08:51.421671+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:51.421671+0000","last_peered":"2023-03-29T14:08:51.421671+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:51.421671+0000","last_undegraded":"2023-03-29T14:08:51.421671+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":331,"reported_epoch":389,"state":"active+undersized","last_fresh":"2023-03-29T14:08:55.311753+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:08:55.311753+0000","last_peered":"2023-03-29T14:08:55.311753+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:08:55.311753+0000","last_undegraded":"2023-03-29T14:08:55.311753+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"385'215","reported_seq":781,"reported_epoch":387,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:52.907437+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:08:52.907437+0000","last_peered":"2023-03-29T14:08:52.907437+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:08:52.907437+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22993164,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":290,"num_read_kb":169953,"num_write":412,"num_write_kb":121700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":76}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":220855662,"num_objects":862,"num_object_clones":770,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3364,"num_read_kb":2050265,"num_write":4566,"num_write_kb":1325120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2455,"ondisk_log_size":2455,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":492616,"kb_used_data":470144,"kb_used_omap":0,"kb_used_meta":22464,"kb_avail":93879224,"statfs":{"total":96636764160,"available":96132325376,"internally_reserved":0,"allocated":481427456,"data_stored":594251941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23003136},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":542872,"kb_used_data":519184,"kb_used_omap":0,"kb_used_meta":23680,"kb_avail":93828968,"statfs":{"total":96636764160,"available":96080863232,"internally_reserved":0,"allocated":531644416,"data_stored":655642842,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24248320},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":490912,"kb_used_data":468120,"kb_used_omap":0,"kb_used_meta":22784,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":479354880,"data_stored":590875982,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23330816},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:00.613 INFO:tasks.rados.rados.0.smithi017.stdout:1592: writing smithi017107884-42 from 1649947 to 2279205 tid 2 2023-03-29T14:09:00.618 INFO:tasks.rados.rados.0.smithi017.stdout:1592: writing smithi017107884-42 from 2834752 to 2924583 tid 3 2023-03-29T14:09:00.618 INFO:tasks.rados.rados.0.smithi017.stdout:1586: done (4 left) 2023-03-29T14:09:00.618 INFO:tasks.rados.rados.0.smithi017.stdout:1593: read oid 30 snap -1 2023-03-29T14:09:00.618 INFO:tasks.rados.rados.0.smithi017.stdout:1593: expect (ObjNum 729 snap 177 seq_num 729) 2023-03-29T14:09:00.618 INFO:tasks.rados.rados.0.smithi017.stdout:1594: read oid 33 snap -1 2023-03-29T14:09:00.618 INFO:tasks.rados.rados.0.smithi017.stdout:1594: expect deleted 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1595: write oid 7 current snap is 188 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1590: finishing write tid 4 to smithi017107884-45 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1590: oid 45 updating version 0 to 209 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1590: oid 45 version 209 is already newer than 209 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1590: oid 45 version 209 is already newer than 209 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 45 v 209 (ObjNum 767 snap 188 seq_num 767) dirty exists 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1590: left oid 45 (ObjNum 767 snap 188 seq_num 767) 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: finishing write tid 2 to smithi017107884-9 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: finishing write tid 3 to smithi017107884-9 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: finishing write tid 4 to smithi017107884-9 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: finishing write tid 5 to smithi017107884-9 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: oid 9 updating version 0 to 248 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: oid 9 version 248 is already newer than 244 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: oid 9 version 248 is already newer than 248 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: oid 9 version 248 is already newer than 248 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 248 (ObjNum 765 snap 188 seq_num 765) dirty exists 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1587: left oid 9 (ObjNum 765 snap 188 seq_num 765) 2023-03-29T14:09:00.619 INFO:tasks.rados.rados.0.smithi017.stdout:1595: seq_num 770 ranges {541390=493383,1592076=489269,2862175=79215} 2023-03-29T14:09:00.626 INFO:tasks.rados.rados.0.smithi017.stdout:1595: writing smithi017107884-7 from 541390 to 1034773 tid 1 2023-03-29T14:09:00.634 INFO:tasks.rados.rados.0.smithi017.stdout:1595: writing smithi017107884-7 from 1592076 to 2081345 tid 2 2023-03-29T14:09:00.639 INFO:tasks.rados.rados.0.smithi017.stdout:1595: writing smithi017107884-7 from 2862175 to 2941390 tid 3 2023-03-29T14:09:00.639 INFO:tasks.rados.rados.0.smithi017.stdout:1587: done (6 left) 2023-03-29T14:09:00.639 INFO:tasks.rados.rados.0.smithi017.stdout:1590: done (5 left) 2023-03-29T14:09:00.639 INFO:tasks.rados.rados.0.smithi017.stdout:1596: write (excl) oid 6 current snap is 188 2023-03-29T14:09:00.639 INFO:tasks.rados.rados.0.smithi017.stdout:1589: finishing write tid 1 to smithi017107884-40 2023-03-29T14:09:00.639 INFO:tasks.rados.rados.0.smithi017.stdout:1596: seq_num 771 ranges {558197=568632,1650558=632927,2773245=184952} 2023-03-29T14:09:00.647 INFO:tasks.rados.rados.0.smithi017.stdout:1596: writing smithi017107884-6 from 558197 to 1126829 tid 1 2023-03-29T14:09:00.657 INFO:tasks.rados.rados.0.smithi017.stdout:1596: writing smithi017107884-6 from 1650558 to 2283485 tid 2 2023-03-29T14:09:00.662 INFO:tasks.rados.rados.0.smithi017.stdout:1596: writing smithi017107884-6 from 2773245 to 2958197 tid 3 2023-03-29T14:09:00.662 INFO:tasks.rados.rados.0.smithi017.stdout:1597: read oid 27 snap -1 2023-03-29T14:09:00.662 INFO:tasks.rados.rados.0.smithi017.stdout:1597: expect deleted 2023-03-29T14:09:00.662 INFO:tasks.rados.rados.0.smithi017.stdout:1598: write (excl) oid 38 current snap is 188 2023-03-29T14:09:00.662 INFO:tasks.rados.rados.0.smithi017.stdout:1589: finishing write tid 2 to smithi017107884-40 2023-03-29T14:09:00.662 INFO:tasks.rados.rados.0.smithi017.stdout:1598: seq_num 772 ranges {575004=643881,1709040=492938,2917021=57983} 2023-03-29T14:09:00.672 INFO:tasks.rados.rados.0.smithi017.stdout:1598: writing smithi017107884-38 from 575004 to 1218885 tid 1 2023-03-29T14:09:00.679 INFO:tasks.rados.rados.0.smithi017.stdout:1598: writing smithi017107884-38 from 1709040 to 2201978 tid 2 2023-03-29T14:09:00.684 INFO:tasks.rados.rados.0.smithi017.stdout:1598: writing smithi017107884-38 from 2917021 to 2975004 tid 3 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1599: delete oid 36 current snap is 188 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: finishing write tid 3 to smithi017107884-40 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: finishing write tid 4 to smithi017107884-40 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: finishing write tid 5 to smithi017107884-40 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: oid 40 updating version 0 to 253 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: oid 40 version 253 is already newer than 253 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: oid 40 version 253 is already newer than 253 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: oid 40 version 253 is already newer than 253 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 253 (ObjNum 766 snap 188 seq_num 766) dirty exists 2023-03-29T14:09:00.685 INFO:tasks.rados.rados.0.smithi017.stdout:1589: left oid 40 (ObjNum 766 snap 188 seq_num 766) 2023-03-29T14:09:00.700 INFO:tasks.rados.rados.0.smithi017.stdout:1595: finishing write tid 1 to smithi017107884-7 2023-03-29T14:09:00.786 INFO:tasks.rados.rados.0.smithi017.stdout:1589: done (8 left) 2023-03-29T14:09:00.786 INFO:tasks.rados.rados.0.smithi017.stdout:1599: done (7 left) 2023-03-29T14:09:00.786 INFO:tasks.rados.rados.0.smithi017.stdout:1600: write oid 50 current snap is 188 2023-03-29T14:09:00.786 INFO:tasks.rados.rados.0.smithi017.stdout:1600: seq_num 774 ranges {608618=794379,2109651=780254} 2023-03-29T14:09:00.796 INFO:tasks.rados.rados.0.smithi017.stdout:1600: writing smithi017107884-50 from 608618 to 1402997 tid 1 2023-03-29T14:09:00.808 INFO:tasks.rados.rados.0.smithi017.stdout:1600: writing smithi017107884-50 from 2109651 to 2889905 tid 2 2023-03-29T14:09:00.809 INFO:tasks.rados.rados.0.smithi017.stdout:1601: read oid 41 snap -1 2023-03-29T14:09:00.809 INFO:tasks.rados.rados.0.smithi017.stdout:1601: expect deleted 2023-03-29T14:09:00.809 INFO:tasks.rados.rados.0.smithi017.stdout:1602: read oid 19 snap -1 2023-03-29T14:09:00.809 INFO:tasks.rados.rados.0.smithi017.stdout:1602: expect (ObjNum 739 snap 178 seq_num 739) 2023-03-29T14:09:00.809 INFO:tasks.rados.rados.0.smithi017.stdout:1603: write (excl) oid 26 current snap is 188 2023-03-29T14:09:00.809 INFO:tasks.rados.rados.0.smithi017.stdout:1596: finishing write tid 1 to smithi017107884-6 2023-03-29T14:09:00.809 INFO:tasks.rados.rados.0.smithi017.stdout:1603: seq_num 775 ranges {625425=469628,1768133=640265} 2023-03-29T14:09:00.816 INFO:tasks.rados.rados.0.smithi017.stdout:1603: writing smithi017107884-26 from 625425 to 1095053 tid 1 2023-03-29T14:09:00.827 INFO:tasks.rados.rados.0.smithi017.stdout:1603: writing smithi017107884-26 from 1768133 to 2408398 tid 2 2023-03-29T14:09:00.827 INFO:tasks.rados.rados.0.smithi017.stdout:1604: snap_remove snap 176 2023-03-29T14:09:01.236 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~231s 2023-03-29T14:09:01.311 INFO:tasks.rados.rados.0.smithi017.stdout:1604: done (11 left) 2023-03-29T14:09:01.311 INFO:tasks.rados.rados.0.smithi017.stdout:1605: read oid 25 snap -1 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1605: expect (ObjNum 742 snap 180 seq_num 742) 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1606: snap_create 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1595: finishing write tid 2 to smithi017107884-7 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: finishing write tid 2 to smithi017107884-6 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: finishing write tid 3 to smithi017107884-6 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: finishing write tid 4 to smithi017107884-6 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: finishing write tid 5 to smithi017107884-6 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: oid 6 updating version 0 to 195 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: oid 6 version 195 is already newer than 195 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: oid 6 version 195 is already newer than 191 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: oid 6 version 195 is already newer than 195 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 195 (ObjNum 771 snap 188 seq_num 771) dirty exists 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1596: left oid 6 (ObjNum 771 snap 188 seq_num 771) 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1598: finishing write tid 1 to smithi017107884-38 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1595: finishing write tid 3 to smithi017107884-7 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1592: finishing write tid 1 to smithi017107884-42 2023-03-29T14:09:01.312 INFO:tasks.rados.rados.0.smithi017.stdout:1603: finishing write tid 1 to smithi017107884-26 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1595: finishing write tid 4 to smithi017107884-7 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1595: finishing write tid 5 to smithi017107884-7 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1595: oid 7 updating version 0 to 88 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1595: oid 7 version 88 is already newer than 88 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1595: oid 7 version 88 is already newer than 88 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1595: oid 7 version 88 is already newer than 88 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 88 (ObjNum 770 snap 188 seq_num 770) dirty exists 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1595: left oid 7 (ObjNum 770 snap 188 seq_num 770) 2023-03-29T14:09:01.313 INFO:tasks.rados.rados.0.smithi017.stdout:1593: expect (ObjNum 729 snap 177 seq_num 729) 2023-03-29T14:09:01.475 INFO:tasks.rados.rados.0.smithi017.stdout:1600: finishing write tid 1 to smithi017107884-50 2023-03-29T14:09:01.475 INFO:tasks.rados.rados.0.smithi017.stdout:1600: finishing write tid 2 to smithi017107884-50 2023-03-29T14:09:01.475 INFO:tasks.rados.rados.0.smithi017.stdout:1602: expect (ObjNum 739 snap 178 seq_num 739) 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1592: finishing write tid 2 to smithi017107884-42 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1603: finishing write tid 2 to smithi017107884-26 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1603: finishing write tid 3 to smithi017107884-26 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1603: finishing write tid 4 to smithi017107884-26 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1603: oid 26 updating version 0 to 198 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1603: oid 26 version 198 is already newer than 198 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1603: oid 26 version 198 is already newer than 196 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 198 (ObjNum 775 snap 188 seq_num 775) dirty exists 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1603: left oid 26 (ObjNum 775 snap 188 seq_num 775) 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1600: finishing write tid 3 to smithi017107884-50 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1600: finishing write tid 4 to smithi017107884-50 2023-03-29T14:09:01.603 INFO:tasks.rados.rados.0.smithi017.stdout:1600: oid 50 updating version 0 to 140 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1600: oid 50 version 140 is already newer than 140 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1600: oid 50 version 140 is already newer than 140 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 140 (ObjNum 774 snap 188 seq_num 774) dirty exists 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1600: left oid 50 (ObjNum 774 snap 188 seq_num 774) 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: finishing write tid 2 to smithi017107884-38 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: finishing write tid 3 to smithi017107884-38 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: finishing write tid 4 to smithi017107884-38 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: finishing write tid 5 to smithi017107884-38 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: oid 38 updating version 0 to 45 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: oid 38 updating version 45 to 49 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: oid 38 version 49 is already newer than 49 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: oid 38 version 49 is already newer than 49 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 49 (ObjNum 772 snap 188 seq_num 772) dirty exists 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1598: left oid 38 (ObjNum 772 snap 188 seq_num 772) 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1592: finishing write tid 3 to smithi017107884-42 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1592: finishing write tid 4 to smithi017107884-42 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1592: finishing write tid 5 to smithi017107884-42 2023-03-29T14:09:01.604 INFO:tasks.rados.rados.0.smithi017.stdout:1592: oid 42 updating version 0 to 258 2023-03-29T14:09:01.605 INFO:tasks.rados.rados.0.smithi017.stdout:1592: oid 42 version 258 is already newer than 254 2023-03-29T14:09:01.605 INFO:tasks.rados.rados.0.smithi017.stdout:1592: oid 42 version 258 is already newer than 258 2023-03-29T14:09:01.605 INFO:tasks.rados.rados.0.smithi017.stdout:1592: oid 42 version 258 is already newer than 258 2023-03-29T14:09:01.605 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 258 (ObjNum 769 snap 188 seq_num 769) dirty exists 2023-03-29T14:09:01.605 INFO:tasks.rados.rados.0.smithi017.stdout:1592: left oid 42 (ObjNum 769 snap 188 seq_num 769) 2023-03-29T14:09:01.605 INFO:tasks.rados.rados.0.smithi017.stdout:1605: expect (ObjNum 742 snap 180 seq_num 742) 2023-03-29T14:09:02.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:02.227+0000 7fd958aac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:02.231 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:02.231+0000 7fd958aac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:02.296 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:02.297 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:02.314 INFO:tasks.rados.rados.0.smithi017.stdout:1592: done (12 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1593: done (11 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1594: done (10 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1595: done (9 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1596: done (8 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1597: done (7 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1598: done (6 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1600: done (5 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1601: done (4 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1602: done (3 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1603: done (2 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1605: done (1 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1606: done (0 left) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1607: read oid 8 snap -1 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1607: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1608: read oid 38 snap 181 2023-03-29T14:09:02.315 INFO:tasks.rados.rados.0.smithi017.stdout:1608: expect (ObjNum 701 snap 170 seq_num 701) 2023-03-29T14:09:02.316 INFO:tasks.rados.rados.0.smithi017.stdout:1609: write (excl) oid 48 current snap is 189 2023-03-29T14:09:02.316 INFO:tasks.rados.rados.0.smithi017.stdout:1609: seq_num 776 ranges {642232=661230,1942968=783923} 2023-03-29T14:09:02.324 INFO:tasks.rados.rados.0.smithi017.stdout:1609: writing smithi017107884-48 from 642232 to 1303462 tid 1 2023-03-29T14:09:02.336 INFO:tasks.rados.rados.0.smithi017.stdout:1609: writing smithi017107884-48 from 1942968 to 2726891 tid 2 2023-03-29T14:09:02.336 INFO:tasks.rados.rados.0.smithi017.stdout:1610: write oid 19 current snap is 189 2023-03-29T14:09:02.336 INFO:tasks.rados.rados.0.smithi017.stdout:1610: seq_num 777 ranges {659039=736479,2001450=643934} 2023-03-29T14:09:02.345 INFO:tasks.rados.rados.0.smithi017.stdout:1610: writing smithi017107884-19 from 659039 to 1395518 tid 1 2023-03-29T14:09:02.352 INFO:tasks.rados.rados.0.smithi017.stdout:1610: writing smithi017107884-19 from 2001450 to 2645384 tid 2 2023-03-29T14:09:02.352 INFO:tasks.rados.rados.0.smithi017.stdout:1611: snap_create 2023-03-29T14:09:02.484 INFO:tasks.rados.rados.0.smithi017.stdout:1607: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:09:02.762 INFO:tasks.rados.rados.0.smithi017.stdout:1609: finishing write tid 1 to smithi017107884-48 2023-03-29T14:09:02.762 INFO:tasks.rados.rados.0.smithi017.stdout:1608: expect (ObjNum 701 snap 170 seq_num 701) 2023-03-29T14:09:02.908 INFO:tasks.rados.rados.0.smithi017.stdout:1610: finishing write tid 1 to smithi017107884-19 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1610: finishing write tid 2 to smithi017107884-19 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1610: finishing write tid 3 to smithi017107884-19 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1609: finishing write tid 2 to smithi017107884-48 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1610: finishing write tid 4 to smithi017107884-19 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1610: oid 19 updating version 0 to 221 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1610: oid 19 version 221 is already newer than 221 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1610: oid 19 version 221 is already newer than 221 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 221 (ObjNum 777 snap 189 seq_num 777) dirty exists 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1610: left oid 19 (ObjNum 777 snap 189 seq_num 777) 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1609: finishing write tid 3 to smithi017107884-48 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1609: finishing write tid 4 to smithi017107884-48 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1609: oid 48 updating version 0 to 250 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1609: oid 48 version 250 is already newer than 247 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1609: oid 48 version 250 is already newer than 250 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 48 v 250 (ObjNum 776 snap 189 seq_num 776) dirty exists 2023-03-29T14:09:02.909 INFO:tasks.rados.rados.0.smithi017.stdout:1609: left oid 48 (ObjNum 776 snap 189 seq_num 776) 2023-03-29T14:09:03.256 INFO:tasks.rados.rados.0.smithi017.stdout:1607: done (4 left) 2023-03-29T14:09:03.256 INFO:tasks.rados.rados.0.smithi017.stdout:1608: done (3 left) 2023-03-29T14:09:03.256 INFO:tasks.rados.rados.0.smithi017.stdout:1609: done (2 left) 2023-03-29T14:09:03.257 INFO:tasks.rados.rados.0.smithi017.stdout:1610: done (1 left) 2023-03-29T14:09:03.257 INFO:tasks.rados.rados.0.smithi017.stdout:1611: done (0 left) 2023-03-29T14:09:03.257 INFO:tasks.rados.rados.0.smithi017.stdout:1612: delete oid 18 current snap is 190 2023-03-29T14:09:03.285 INFO:tasks.rados.rados.0.smithi017.stdout:1612: done (0 left) 2023-03-29T14:09:03.285 INFO:tasks.rados.rados.0.smithi017.stdout:1613: read oid 8 snap -1 2023-03-29T14:09:03.285 INFO:tasks.rados.rados.0.smithi017.stdout:1613: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:09:03.285 INFO:tasks.rados.rados.0.smithi017.stdout:1614: read oid 24 snap -1 2023-03-29T14:09:03.285 INFO:tasks.rados.rados.0.smithi017.stdout:1614: expect deleted 2023-03-29T14:09:03.285 INFO:tasks.rados.rados.0.smithi017.stdout:1615: delete oid 28 current snap is 190 2023-03-29T14:09:03.286 INFO:tasks.rados.rados.0.smithi017.stdout:1615: done (2 left) 2023-03-29T14:09:03.286 INFO:tasks.rados.rados.0.smithi017.stdout:1616: delete oid 15 current snap is 190 2023-03-29T14:09:03.391 INFO:tasks.rados.rados.0.smithi017.stdout:1614: done (2 left) 2023-03-29T14:09:03.391 INFO:tasks.rados.rados.0.smithi017.stdout:1616: done (1 left) 2023-03-29T14:09:03.391 INFO:tasks.rados.rados.0.smithi017.stdout:1617: write oid 16 current snap is 190 2023-03-29T14:09:03.391 INFO:tasks.rados.rados.0.smithi017.stdout:1617: seq_num 781 ranges {726267=637475,2119025=651272} 2023-03-29T14:09:03.398 INFO:tasks.rados.rados.0.smithi017.stdout:1617: writing smithi017107884-16 from 726267 to 1363742 tid 1 2023-03-29T14:09:03.406 INFO:tasks.rados.rados.0.smithi017.stdout:1617: writing smithi017107884-16 from 2119025 to 2770297 tid 2 2023-03-29T14:09:03.406 INFO:tasks.rados.rados.0.smithi017.stdout:1618: read oid 23 snap -1 2023-03-29T14:09:03.406 INFO:tasks.rados.rados.0.smithi017.stdout:1618: expect (ObjNum 719 snap 177 seq_num 719) 2023-03-29T14:09:03.406 INFO:tasks.rados.rados.0.smithi017.stdout:1619: write (excl) oid 24 current snap is 190 2023-03-29T14:09:03.406 INFO:tasks.rados.rados.0.smithi017.stdout:1619: seq_num 782 ranges {743074=712724,2061154=794930} 2023-03-29T14:09:03.414 INFO:tasks.rados.rados.0.smithi017.stdout:1619: writing smithi017107884-24 from 743074 to 1455798 tid 1 2023-03-29T14:09:03.422 INFO:tasks.rados.rados.0.smithi017.stdout:1619: writing smithi017107884-24 from 2061154 to 2856084 tid 2 2023-03-29T14:09:03.422 INFO:tasks.rados.rados.0.smithi017.stdout:1620: write oid 20 current snap is 190 2023-03-29T14:09:03.422 INFO:tasks.rados.rados.0.smithi017.stdout:1620: seq_num 783 ranges {759881=787973,2119636=654941} 2023-03-29T14:09:03.430 INFO:tasks.rados.rados.0.smithi017.stdout:1620: writing smithi017107884-20 from 759881 to 1547854 tid 1 2023-03-29T14:09:03.437 INFO:tasks.rados.rados.0.smithi017.stdout:1620: writing smithi017107884-20 from 2119636 to 2774577 tid 2 2023-03-29T14:09:03.437 INFO:tasks.rados.rados.0.smithi017.stdout:1621: delete oid 33 current snap is 190 2023-03-29T14:09:03.437 INFO:tasks.rados.rados.0.smithi017.stdout:1613: expect (ObjNum 589 snap 139 seq_num 589) 2023-03-29T14:09:03.724 INFO:tasks.rados.rados.0.smithi017.stdout:1617: finishing write tid 1 to smithi017107884-16 2023-03-29T14:09:03.724 INFO:tasks.rados.rados.0.smithi017.stdout:1618: expect (ObjNum 719 snap 177 seq_num 719) 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1617: finishing write tid 2 to smithi017107884-16 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1617: finishing write tid 3 to smithi017107884-16 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1617: finishing write tid 4 to smithi017107884-16 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1617: oid 16 updating version 0 to 124 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1617: oid 16 version 124 is already newer than 124 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1617: oid 16 version 124 is already newer than 124 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 124 (ObjNum 781 snap 190 seq_num 781) dirty exists 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1617: left oid 16 (ObjNum 781 snap 190 seq_num 781) 2023-03-29T14:09:03.805 INFO:tasks.rados.rados.0.smithi017.stdout:1620: finishing write tid 1 to smithi017107884-20 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1620: finishing write tid 2 to smithi017107884-20 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1620: finishing write tid 3 to smithi017107884-20 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1620: finishing write tid 4 to smithi017107884-20 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1620: oid 20 updating version 0 to 213 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1620: oid 20 version 213 is already newer than 213 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1620: oid 20 version 213 is already newer than 213 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 213 (ObjNum 783 snap 190 seq_num 783) dirty exists 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1620: left oid 20 (ObjNum 783 snap 190 seq_num 783) 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: finishing write tid 1 to smithi017107884-24 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: finishing write tid 2 to smithi017107884-24 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: finishing write tid 3 to smithi017107884-24 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: finishing write tid 4 to smithi017107884-24 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: oid 24 updating version 0 to 98 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: oid 24 version 98 is already newer than 98 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: oid 24 version 98 is already newer than 96 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 98 (ObjNum 782 snap 190 seq_num 782) dirty exists 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1619: left oid 24 (ObjNum 782 snap 190 seq_num 782) 2023-03-29T14:09:03.806 INFO:tasks.rados.rados.0.smithi017.stdout:1613: done (5 left) 2023-03-29T14:09:03.807 INFO:tasks.rados.rados.0.smithi017.stdout:1617: done (4 left) 2023-03-29T14:09:03.807 INFO:tasks.rados.rados.0.smithi017.stdout:1618: done (3 left) 2023-03-29T14:09:03.807 INFO:tasks.rados.rados.0.smithi017.stdout:1619: done (2 left) 2023-03-29T14:09:03.807 INFO:tasks.rados.rados.0.smithi017.stdout:1620: done (1 left) 2023-03-29T14:09:03.807 INFO:tasks.rados.rados.0.smithi017.stdout:1621: done (0 left) 2023-03-29T14:09:03.807 INFO:tasks.rados.rados.0.smithi017.stdout:1622: snap_remove snap 185 2023-03-29T14:09:04.297 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:04.321 INFO:tasks.rados.rados.0.smithi017.stdout:1622: done (0 left) 2023-03-29T14:09:04.321 INFO:tasks.rados.rados.0.smithi017.stdout:1623: read oid 5 snap 174 2023-03-29T14:09:04.321 INFO:tasks.rados.rados.0.smithi017.stdout:1623: expect (ObjNum 555 snap 129 seq_num 555) 2023-03-29T14:09:04.321 INFO:tasks.rados.rados.0.smithi017.stdout:1624: snap_remove snap 187 2023-03-29T14:09:04.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:04.428+0000 7f01fd8ba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:04.432+0000 7f01fd8ba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:04.493 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:04.495 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:04.496 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:05.322 INFO:tasks.rados.rados.0.smithi017.stdout:1624: done (1 left) 2023-03-29T14:09:05.322 INFO:tasks.rados.rados.0.smithi017.stdout:1625: snap_remove snap 186 2023-03-29T14:09:05.609 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:05.610 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:05.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:05.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:05.738+0000 7fc959fd4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:05.745+0000 7fc959fd4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:05.961 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2023-03-29T14:09:05.320656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221633360,"num_objects":892,"num_object_clones":796,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3586,"num_read_kb":2093412,"num_write":4960,"num_write_kb":1360388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2622,"ondisk_log_size":2622,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1708676,"kb_used_data":1627044,"kb_used_omap":0,"kb_used_meta":81600,"kb_avail":375778684,"statfs":{"total":386547056640,"available":384797372416,"internally_reserved":0,"allocated":1666093056,"data_stored":2048847160,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":83558400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4067954,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":49,"num_read_kb":18822,"num_write":112,"num_write_kb":29074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.011401"},"pg_stats":[{"pgid":"3.f","version":"386'120","reported_seq":375,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:00.342978+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:00.342978+0000","last_peered":"2023-03-29T14:09:00.342978+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:00.342978+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13076440,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":65944,"num_write":154,"num_write_kb":44762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":24}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"393'49","reported_seq":404,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:00.910686+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:00.910686+0000","last_peered":"2023-03-29T14:09:00.910686+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:00.910686+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5950008,"num_objects":20,"num_object_clones":18,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":25433,"num_write":92,"num_write_kb":27604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"393'246","reported_seq":881,"reported_epoch":394,"state":"active+clean","last_fresh":"2023-03-29T14:09:00.478118+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:00.478118+0000","last_peered":"2023-03-29T14:09:00.478118+0000","last_clean":"2023-03-29T14:09:00.478118+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:00.478118+0000","last_undegraded":"2023-03-29T14:09:00.478118+0000","last_fullsized":"2023-03-29T14:09:00.478118+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28976720,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":359,"num_read_kb":251014,"num_write":476,"num_write_kb":140782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"393'258","reported_seq":898,"reported_epoch":394,"state":"active+clean","last_fresh":"2023-03-29T14:09:01.348043+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:01.348043+0000","last_peered":"2023-03-29T14:09:01.348043+0000","last_clean":"2023-03-29T14:09:01.348043+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:01.348043+0000","last_undegraded":"2023-03-29T14:09:01.348043+0000","last_fullsized":"2023-03-29T14:09:01.348043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253788,"num_objects":102,"num_object_clones":94,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":354,"num_read_kb":212496,"num_write":506,"num_write_kb":142676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"393'209","reported_seq":769,"reported_epoch":393,"state":"active+clean","last_fresh":"2023-03-29T14:09:00.487280+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:00.487280+0000","last_peered":"2023-03-29T14:09:00.487280+0000","last_clean":"2023-03-29T14:09:00.487280+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:00.487280+0000","last_undegraded":"2023-03-29T14:09:00.487280+0000","last_fullsized":"2023-03-29T14:09:00.487280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23598216,"num_objects":74,"num_object_clones":66,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":287,"num_read_kb":180236,"num_write":416,"num_write_kb":122046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":332,"reported_epoch":394,"state":"active+undersized","last_fresh":"2023-03-29T14:09:00.345350+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:00.345350+0000","last_peered":"2023-03-29T14:09:00.345350+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:00.345350+0000","last_undegraded":"2023-03-29T14:09:00.345350+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"393'141","reported_seq":626,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:00.439670+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:00.439670+0000","last_peered":"2023-03-29T14:09:00.439670+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:00.439670+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9681492,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":74213,"num_write":256,"num_write_kb":71774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":50}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"393'88","reported_seq":518,"reported_epoch":394,"state":"active+clean","last_fresh":"2023-03-29T14:09:00.941803+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:00.941803+0000","last_peered":"2023-03-29T14:09:00.941803+0000","last_clean":"2023-03-29T14:09:00.941803+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:00.941803+0000","last_undegraded":"2023-03-29T14:09:00.941803+0000","last_fullsized":"2023-03-29T14:09:00.941803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5882780,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":113,"num_read_kb":57968,"num_write":162,"num_write_kb":45532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":306,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-03-29T14:08:59.355571+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:08:59.355571+0000","last_peered":"2023-03-29T14:08:59.355571+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:08:59.355571+0000","last_undegraded":"2023-03-29T14:08:59.355571+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"390'98","reported_seq":296,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:00.924033+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:00.924033+0000","last_peered":"2023-03-29T14:09:00.924033+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:00.924033+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084860,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":18402,"num_write":120,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":335,"reported_epoch":394,"state":"active+undersized","last_fresh":"2023-03-29T14:09:00.338474+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:00.338474+0000","last_peered":"2023-03-29T14:09:00.338474+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:00.338474+0000","last_undegraded":"2023-03-29T14:09:00.338474+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"382'95","reported_seq":532,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:58.530242+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:08:58.530242+0000","last_peered":"2023-03-29T14:08:58.530242+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:08:58.530242+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":158,"num_read_kb":107602,"num_write":182,"num_write_kb":52894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-03-29T14:08:59.339270+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:08:59.339270+0000","last_peered":"2023-03-29T14:08:59.339270+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:08:59.339270+0000","last_undegraded":"2023-03-29T14:08:59.339270+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"393'140","reported_seq":624,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:00.885041+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:00.885041+0000","last_peered":"2023-03-29T14:09:00.885041+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:00.885041+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10824302,"num_objects":52,"num_object_clones":46,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":219,"num_read_kb":130738,"num_write":268,"num_write_kb":79606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":335,"reported_epoch":394,"state":"active+undersized","last_fresh":"2023-03-29T14:09:00.334611+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:00.334611+0000","last_peered":"2023-03-29T14:09:00.334611+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:00.334611+0000","last_undegraded":"2023-03-29T14:09:00.334611+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":531,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:59.348560+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:08:59.348560+0000","last_peered":"2023-03-29T14:08:59.348560+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:08:59.348560+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"393'198","reported_seq":815,"reported_epoch":394,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:00.991251+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:00.991251+0000","last_peered":"2023-03-29T14:09:00.991251+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:00.991251+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19900412,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":354,"num_read_kb":239220,"num_write":384,"num_write_kb":111100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":72}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":335,"reported_epoch":394,"state":"active+undersized","last_fresh":"2023-03-29T14:09:00.333869+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:00.333869+0000","last_peered":"2023-03-29T14:09:00.333869+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:00.333869+0000","last_undegraded":"2023-03-29T14:09:00.333869+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"382'325","reported_seq":1055,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:08:58.537389+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:08:58.537389+0000","last_peered":"2023-03-29T14:08:58.537389+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:08:58.537389+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31498166,"num_objects":114,"num_object_clones":100,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":511,"num_read_kb":314596,"num_write":612,"num_write_kb":176924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":114}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":306,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-03-29T14:08:59.352553+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:08:59.352553+0000","last_peered":"2023-03-29T14:08:59.352553+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:08:59.352553+0000","last_undegraded":"2023-03-29T14:08:59.352553+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"372'116","reported_seq":552,"reported_epoch":393,"state":"active+clean","last_fresh":"2023-03-29T14:08:59.340575+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:08:59.340575+0000","last_peered":"2023-03-29T14:08:59.340575+0000","last_clean":"2023-03-29T14:08:59.340575+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:08:59.340575+0000","last_undegraded":"2023-03-29T14:08:59.340575+0000","last_fullsized":"2023-03-29T14:08:59.340575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6823906,"num_objects":42,"num_object_clones":38,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":142,"num_read_kb":79797,"num_write":234,"num_write_kb":68058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"384'225","reported_seq":777,"reported_epoch":393,"state":"active+clean","last_fresh":"2023-03-29T14:08:59.344668+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:08:59.344668+0000","last_peered":"2023-03-29T14:08:59.344668+0000","last_clean":"2023-03-29T14:08:59.344668+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:08:59.344668+0000","last_undegraded":"2023-03-29T14:08:59.344668+0000","last_fullsized":"2023-03-29T14:08:59.344668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14219712,"num_objects":80,"num_object_clones":70,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":272,"num_read_kb":159210,"num_write":412,"num_write_kb":119550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":306,"reported_epoch":393,"state":"active+undersized","last_fresh":"2023-03-29T14:08:59.334657+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:08:59.334657+0000","last_peered":"2023-03-29T14:08:59.334657+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:08:59.334657+0000","last_undegraded":"2023-03-29T14:08:59.334657+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":335,"reported_epoch":394,"state":"active+undersized","last_fresh":"2023-03-29T14:09:00.340158+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:00.340158+0000","last_peered":"2023-03-29T14:09:00.340158+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:00.340158+0000","last_undegraded":"2023-03-29T14:09:00.340158+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"390'217","reported_seq":790,"reported_epoch":393,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:00.952626+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:00.952626+0000","last_peered":"2023-03-29T14:09:00.952626+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:00.952626+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17681822,"num_objects":78,"num_object_clones":70,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":78,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":297,"num_read_kb":176423,"num_write":412,"num_write_kb":121700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":78}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":220452624,"num_objects":888,"num_object_clones":796,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3444,"num_read_kb":2093292,"num_write":4686,"num_write_kb":1356600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":504864,"kb_used_data":482136,"kb_used_omap":0,"kb_used_meta":22720,"kb_avail":93866976,"statfs":{"total":96636764160,"available":96119783424,"internally_reserved":0,"allocated":493707264,"data_stored":610437319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23265280},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":549408,"kb_used_data":525592,"kb_used_omap":0,"kb_used_meta":23808,"kb_avail":93822432,"statfs":{"total":96636764160,"available":96074170368,"internally_reserved":0,"allocated":538206208,"data_stored":664640170,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24379392},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":498460,"kb_used_data":475476,"kb_used_omap":0,"kb_used_meta":22976,"kb_avail":93873380,"statfs":{"total":96636764160,"available":96126341120,"internally_reserved":0,"allocated":486887424,"data_stored":600537446,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23527424},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:05.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.3 3.2 3.1 2023-03-29T14:09:06.100 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:06.091+0000 7f80ac327700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:06.102+0000 7f80ac327700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:06.307 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3] on osd.0 to force-backfill; instructing pg(s) [3.2,3.f] on osd.3 to force-backfill; 2023-03-29T14:09:06.307 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:09:06.328 INFO:tasks.rados.rados.0.smithi017.stdout:1625: done (1 left) 2023-03-29T14:09:06.328 INFO:tasks.rados.rados.0.smithi017.stdout:1626: write (excl) oid 20 current snap is 190 2023-03-29T14:09:06.328 INFO:tasks.rados.rados.0.smithi017.stdout:1626: seq_num 785 ranges {793495=654824,1952953=658610,3157167=36328} 2023-03-29T14:09:06.336 INFO:tasks.rados.rados.0.smithi017.stdout:1626: writing smithi017107884-20 from 793495 to 1448319 tid 1 2023-03-29T14:09:06.342 INFO:tasks.rados.rados.0.smithi017.stdout:1626: writing smithi017107884-20 from 1952953 to 2611563 tid 2 2023-03-29T14:09:06.346 INFO:tasks.rados.rados.0.smithi017.stdout:1626: writing smithi017107884-20 from 3157167 to 3193495 tid 3 2023-03-29T14:09:06.346 INFO:tasks.rados.rados.0.smithi017.stdout:1627: snap_remove snap 189 2023-03-29T14:09:06.496 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:06.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:06.626+0000 7fc2a5414700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:06.632 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:06.631+0000 7fc2a5414700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:06.717 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:06.719 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:06.720 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:06.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~237s 2023-03-29T14:09:07.329 INFO:tasks.rados.rados.0.smithi017.stdout:1627: done (2 left) 2023-03-29T14:09:07.330 INFO:tasks.rados.rados.0.smithi017.stdout:1628: read oid 39 snap -1 2023-03-29T14:09:07.330 INFO:tasks.rados.rados.0.smithi017.stdout:1628: expect (ObjNum 673 snap 164 seq_num 673) 2023-03-29T14:09:07.330 INFO:tasks.rados.rados.0.smithi017.stdout:1629: snap_create 2023-03-29T14:09:07.330 INFO:tasks.rados.rados.0.smithi017.stdout:1623: expect (ObjNum 555 snap 129 seq_num 555) 2023-03-29T14:09:07.566 INFO:tasks.rados.rados.0.smithi017.stdout:1626: finishing write tid 1 to smithi017107884-20 2023-03-29T14:09:07.566 INFO:tasks.rados.rados.0.smithi017.stdout:1626: finishing write tid 2 to smithi017107884-20 2023-03-29T14:09:07.566 INFO:tasks.rados.rados.0.smithi017.stdout:1626: finishing write tid 3 to smithi017107884-20 2023-03-29T14:09:07.566 INFO:tasks.rados.rados.0.smithi017.stdout:1626: finishing write tid 4 to smithi017107884-20 2023-03-29T14:09:07.566 INFO:tasks.rados.rados.0.smithi017.stdout:1626: finishing write tid 5 to smithi017107884-20 2023-03-29T14:09:07.566 INFO:tasks.rados.rados.0.smithi017.stdout:1626: oid 20 updating version 0 to 217 2023-03-29T14:09:07.567 INFO:tasks.rados.rados.0.smithi017.stdout:1626: oid 20 version 217 is already newer than 214 2023-03-29T14:09:07.567 INFO:tasks.rados.rados.0.smithi017.stdout:1626: oid 20 version 217 is already newer than 217 2023-03-29T14:09:07.567 INFO:tasks.rados.rados.0.smithi017.stdout:1626: oid 20 version 217 is already newer than 217 2023-03-29T14:09:07.567 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 217 (ObjNum 785 snap 190 seq_num 785) dirty exists 2023-03-29T14:09:07.567 INFO:tasks.rados.rados.0.smithi017.stdout:1626: left oid 20 (ObjNum 785 snap 190 seq_num 785) 2023-03-29T14:09:07.567 INFO:tasks.rados.rados.0.smithi017.stdout:1628: expect (ObjNum 673 snap 164 seq_num 673) 2023-03-29T14:09:08.335 INFO:tasks.rados.rados.0.smithi017.stdout:1623: done (3 left) 2023-03-29T14:09:08.335 INFO:tasks.rados.rados.0.smithi017.stdout:1626: done (2 left) 2023-03-29T14:09:08.335 INFO:tasks.rados.rados.0.smithi017.stdout:1628: done (1 left) 2023-03-29T14:09:08.335 INFO:tasks.rados.rados.0.smithi017.stdout:1629: done (0 left) 2023-03-29T14:09:08.335 INFO:tasks.rados.rados.0.smithi017.stdout:1630: snap_remove snap 181 2023-03-29T14:09:08.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:08.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:08.848+0000 7f98c75fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:08.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:08.853+0000 7f98c75fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:08.915 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:08.918 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:08.919 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:09.336 INFO:tasks.rados.rados.0.smithi017.stdout:1630: done (0 left) 2023-03-29T14:09:09.336 INFO:tasks.rados.rados.0.smithi017.stdout:1631: write (excl) oid 35 current snap is 191 2023-03-29T14:09:09.336 INFO:tasks.rados.rados.0.smithi017.stdout:1631: seq_num 786 ranges {410302=730073,1895082=402268,2951884=258418} 2023-03-29T14:09:09.342 INFO:tasks.rados.rados.0.smithi017.stdout:1631: writing smithi017107884-35 from 410302 to 1140375 tid 1 2023-03-29T14:09:09.348 INFO:tasks.rados.rados.0.smithi017.stdout:1631: writing smithi017107884-35 from 1895082 to 2297350 tid 2 2023-03-29T14:09:09.354 INFO:tasks.rados.rados.0.smithi017.stdout:1631: writing smithi017107884-35 from 2951884 to 3210302 tid 3 2023-03-29T14:09:09.354 INFO:tasks.rados.rados.0.smithi017.stdout:1632: write (excl) oid 42 current snap is 191 2023-03-29T14:09:09.354 INFO:tasks.rados.rados.0.smithi017.stdout:1632: seq_num 787 ranges {427109=405322,1553564=545926,2579307=647802} 2023-03-29T14:09:09.359 INFO:tasks.rados.rados.0.smithi017.stdout:1632: writing smithi017107884-42 from 427109 to 832431 tid 1 2023-03-29T14:09:09.366 INFO:tasks.rados.rados.0.smithi017.stdout:1632: writing smithi017107884-42 from 1553564 to 2099490 tid 2 2023-03-29T14:09:09.372 INFO:tasks.rados.rados.0.smithi017.stdout:1632: writing smithi017107884-42 from 2579307 to 3227109 tid 3 2023-03-29T14:09:09.372 INFO:tasks.rados.rados.0.smithi017.stdout:1633: write (excl) oid 9 current snap is 191 2023-03-29T14:09:09.372 INFO:tasks.rados.rados.0.smithi017.stdout:1633: seq_num 788 ranges {443916=480571,1612046=405937,2606730=495052} 2023-03-29T14:09:09.377 INFO:tasks.rados.rados.0.smithi017.stdout:1633: writing smithi017107884-9 from 443916 to 924487 tid 1 2023-03-29T14:09:09.383 INFO:tasks.rados.rados.0.smithi017.stdout:1633: writing smithi017107884-9 from 1612046 to 2017983 tid 2 2023-03-29T14:09:09.390 INFO:tasks.rados.rados.0.smithi017.stdout:1633: writing smithi017107884-9 from 2606730 to 3101782 tid 3 2023-03-29T14:09:09.390 INFO:tasks.rados.rados.0.smithi017.stdout:1634: snap_remove snap 190 2023-03-29T14:09:10.346 INFO:tasks.rados.rados.0.smithi017.stdout:1634: done (3 left) 2023-03-29T14:09:10.346 INFO:tasks.rados.rados.0.smithi017.stdout:1635: write oid 49 current snap is 191 2023-03-29T14:09:10.346 INFO:tasks.rados.rados.0.smithi017.stdout:1631: finishing write tid 1 to smithi017107884-35 2023-03-29T14:09:10.347 INFO:tasks.rados.rados.0.smithi017.stdout:1635: seq_num 789 ranges {460723=555820,1670528=549595,2634153=626570} 2023-03-29T14:09:10.347 INFO:tasks.rados.rados.0.smithi017.stdout:1635: writing smithi017107884-49 from 460723 to 1016543 tid 1 2023-03-29T14:09:10.354 INFO:tasks.rados.rados.0.smithi017.stdout:1635: writing smithi017107884-49 from 1670528 to 2220123 tid 2 2023-03-29T14:09:10.359 INFO:tasks.rados.rados.0.smithi017.stdout:1635: writing smithi017107884-49 from 2634153 to 3260723 tid 3 2023-03-29T14:09:10.360 INFO:tasks.rados.rados.0.smithi017.stdout:1636: snap_remove snap 182 2023-03-29T14:09:10.360 INFO:tasks.rados.rados.0.smithi017.stdout:1632: finishing write tid 1 to smithi017107884-42 2023-03-29T14:09:10.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:11.049 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:11.046+0000 7f090de69700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:11.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:11.051+0000 7f090de69700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:11.113 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:11.116 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:11.117 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:11.320 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:11.320 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:11.320 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:11.348 INFO:tasks.rados.rados.0.smithi017.stdout:1636: done (4 left) 2023-03-29T14:09:11.348 INFO:tasks.rados.rados.0.smithi017.stdout:1637: read oid 7 snap -1 2023-03-29T14:09:11.348 INFO:tasks.rados.rados.0.smithi017.stdout:1637: expect (ObjNum 770 snap 188 seq_num 770) 2023-03-29T14:09:11.348 INFO:tasks.rados.rados.0.smithi017.stdout:1638: write oid 3 current snap is 191 2023-03-29T14:09:11.348 INFO:tasks.rados.rados.0.smithi017.stdout:1631: finishing write tid 2 to smithi017107884-35 2023-03-29T14:09:11.348 INFO:tasks.rados.rados.0.smithi017.stdout:1631: finishing write tid 3 to smithi017107884-35 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: finishing write tid 2 to smithi017107884-42 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: finishing write tid 3 to smithi017107884-42 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: finishing write tid 4 to smithi017107884-42 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: finishing write tid 5 to smithi017107884-42 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: oid 42 updating version 0 to 263 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: oid 42 version 263 is already newer than 259 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: oid 42 version 263 is already newer than 263 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: oid 42 version 263 is already newer than 263 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 263 (ObjNum 787 snap 191 seq_num 787) dirty exists 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1632: left oid 42 (ObjNum 787 snap 191 seq_num 787) 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1631: finishing write tid 4 to smithi017107884-35 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1631: finishing write tid 5 to smithi017107884-35 2023-03-29T14:09:11.349 INFO:tasks.rados.rados.0.smithi017.stdout:1631: oid 35 updating version 0 to 121 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1631: oid 35 version 121 is already newer than 121 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1631: oid 35 version 121 is already newer than 121 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1631: oid 35 version 121 is already newer than 117 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 121 (ObjNum 786 snap 191 seq_num 786) dirty exists 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1631: left oid 35 (ObjNum 786 snap 191 seq_num 786) 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1633: finishing write tid 1 to smithi017107884-9 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1633: finishing write tid 2 to smithi017107884-9 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1633: finishing write tid 3 to smithi017107884-9 2023-03-29T14:09:11.350 INFO:tasks.rados.rados.0.smithi017.stdout:1638: seq_num 790 ranges {477530=631069,1612657=409606,2545223=517596} 2023-03-29T14:09:11.353 INFO:tasks.rados.rados.0.smithi017.stdout:1638: writing smithi017107884-3 from 477530 to 1108599 tid 1 2023-03-29T14:09:11.358 INFO:tasks.rados.rados.0.smithi017.stdout:1638: writing smithi017107884-3 from 1612657 to 2022263 tid 2 2023-03-29T14:09:11.364 INFO:tasks.rados.rados.0.smithi017.stdout:1638: writing smithi017107884-3 from 2545223 to 3062819 tid 3 2023-03-29T14:09:11.364 INFO:tasks.rados.rados.0.smithi017.stdout:1631: done (5 left) 2023-03-29T14:09:11.364 INFO:tasks.rados.rados.0.smithi017.stdout:1632: done (4 left) 2023-03-29T14:09:11.365 INFO:tasks.rados.rados.0.smithi017.stdout:1639: write (excl) oid 38 current snap is 191 2023-03-29T14:09:11.365 INFO:tasks.rados.rados.0.smithi017.stdout:1639: seq_num 791 ranges {494337=422671,1387492=553264,2688999=445221} 2023-03-29T14:09:11.369 INFO:tasks.rados.rados.0.smithi017.stdout:1639: writing smithi017107884-38 from 494337 to 917008 tid 1 2023-03-29T14:09:11.375 INFO:tasks.rados.rados.0.smithi017.stdout:1639: writing smithi017107884-38 from 1387492 to 1940756 tid 2 2023-03-29T14:09:11.382 INFO:tasks.rados.rados.0.smithi017.stdout:1639: writing smithi017107884-38 from 2688999 to 3134220 tid 3 2023-03-29T14:09:11.382 INFO:tasks.rados.rados.0.smithi017.stdout:1640: write (excl) oid 8 current snap is 191 2023-03-29T14:09:11.382 INFO:tasks.rados.rados.0.smithi017.stdout:1633: finishing write tid 4 to smithi017107884-9 2023-03-29T14:09:11.382 INFO:tasks.rados.rados.0.smithi017.stdout:1640: seq_num 792 ranges {511144=497920,1445974=413275,2316422=656493} 2023-03-29T14:09:11.388 INFO:tasks.rados.rados.0.smithi017.stdout:1640: writing smithi017107884-8 from 511144 to 1009064 tid 1 2023-03-29T14:09:11.393 INFO:tasks.rados.rados.0.smithi017.stdout:1640: writing smithi017107884-8 from 1445974 to 1859249 tid 2 2023-03-29T14:09:11.400 INFO:tasks.rados.rados.0.smithi017.stdout:1640: writing smithi017107884-8 from 2316422 to 2972915 tid 3 2023-03-29T14:09:11.400 INFO:tasks.rados.rados.0.smithi017.stdout:1641: read oid 43 snap -1 2023-03-29T14:09:11.400 INFO:tasks.rados.rados.0.smithi017.stdout:1641: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:11.400 INFO:tasks.rados.rados.0.smithi017.stdout:1642: snap_create 2023-03-29T14:09:11.400 INFO:tasks.rados.rados.0.smithi017.stdout:1633: finishing write tid 5 to smithi017107884-9 2023-03-29T14:09:11.400 INFO:tasks.rados.rados.0.smithi017.stdout:1633: oid 9 updating version 0 to 268 2023-03-29T14:09:11.400 INFO:tasks.rados.rados.0.smithi017.stdout:1633: oid 9 version 268 is already newer than 268 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:1633: oid 9 version 268 is already newer than 264 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:1633: oid 9 version 268 is already newer than 268 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 268 (ObjNum 788 snap 191 seq_num 788) dirty exists 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:1633: left oid 9 (ObjNum 788 snap 191 seq_num 788) 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:1635: finishing write tid 1 to smithi017107884-49 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:1635: finishing write tid 2 to smithi017107884-49 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:1635: finishing write tid 3 to smithi017107884-49 2023-03-29T14:09:11.401 INFO:tasks.rados.rados.0.smithi017.stdout:1635: finishing write tid 4 to smithi017107884-49 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:1635: finishing write tid 5 to smithi017107884-49 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:1635: oid 49 updating version 0 to 332 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:1635: oid 49 version 332 is already newer than 332 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:1635: oid 49 version 332 is already newer than 332 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:1635: oid 49 version 332 is already newer than 332 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 332 (ObjNum 789 snap 191 seq_num 789) dirty exists 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:1635: left oid 49 (ObjNum 789 snap 191 seq_num 789) 2023-03-29T14:09:11.402 INFO:tasks.rados.rados.0.smithi017.stdout:1637: expect (ObjNum 770 snap 188 seq_num 770) 2023-03-29T14:09:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:11.454+0000 7f137e3cb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:11.462 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:11.461+0000 7f137e3cb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:11.617 INFO:tasks.rados.rados.0.smithi017.stdout:1638: finishing write tid 1 to smithi017107884-3 2023-03-29T14:09:11.617 INFO:tasks.rados.rados.0.smithi017.stdout:1638: finishing write tid 2 to smithi017107884-3 2023-03-29T14:09:11.617 INFO:tasks.rados.rados.0.smithi017.stdout:1641: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:11.666 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:11.668 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:11.681 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2023-03-29T14:09:11.345759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224860502,"num_objects":906,"num_object_clones":810,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3639,"num_read_kb":2128643,"num_write":5030,"num_write_kb":1382940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2657,"ondisk_log_size":2657,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1750324,"kb_used_data":1667860,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":375737036,"statfs":{"total":386547056640,"available":384754724864,"internally_reserved":0,"allocated":1707888640,"data_stored":2108615299,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1714380,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":30,"num_write_kb":8742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017920"},"pg_stats":[{"pgid":"3.f","version":"396'124","reported_seq":387,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:05.335914+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:05.335914+0000","last_peered":"2023-03-29T14:09:05.335914+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:05.335914+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13950404,"num_objects":26,"num_object_clones":26,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":65945,"num_write":162,"num_write_kb":47282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":26}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"393'49","reported_seq":414,"reported_epoch":403,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:09.477413+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:09.477413+0000","last_peered":"2023-03-29T14:09:09.477413+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:09.477413+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5950008,"num_objects":20,"num_object_clones":18,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":30740,"num_write":92,"num_write_kb":27604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"395'250","reported_seq":893,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-03-29T14:09:05.335357+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:05.335357+0000","last_peered":"2023-03-29T14:09:05.335357+0000","last_clean":"2023-03-29T14:09:05.335357+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:05.335357+0000","last_undegraded":"2023-03-29T14:09:05.335357+0000","last_fullsized":"2023-03-29T14:09:05.335357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27901138,"num_objects":90,"num_object_clones":80,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":362,"num_read_kb":251015,"num_write":484,"num_write_kb":143606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"393'258","reported_seq":903,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-03-29T14:09:05.343583+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:05.343583+0000","last_peered":"2023-03-29T14:09:05.343583+0000","last_clean":"2023-03-29T14:09:05.343583+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:05.343583+0000","last_undegraded":"2023-03-29T14:09:05.343583+0000","last_fullsized":"2023-03-29T14:09:05.343583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253788,"num_objects":102,"num_object_clones":94,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":354,"num_read_kb":212496,"num_write":506,"num_write_kb":142676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"399'217","reported_seq":792,"reported_epoch":403,"state":"active+clean","last_fresh":"2023-03-29T14:09:09.455620+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:09.455620+0000","last_peered":"2023-03-29T14:09:09.455620+0000","last_clean":"2023-03-29T14:09:09.455620+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:09.455620+0000","last_undegraded":"2023-03-29T14:09:09.455620+0000","last_fullsized":"2023-03-29T14:09:09.455620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22489020,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":291,"num_read_kb":180238,"num_write":434,"num_write_kb":127506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":337,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-03-29T14:09:05.342742+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:05.342742+0000","last_peered":"2023-03-29T14:09:05.342742+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:05.342742+0000","last_undegraded":"2023-03-29T14:09:05.342742+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"393'141","reported_seq":632,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:05.344141+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:05.344141+0000","last_peered":"2023-03-29T14:09:05.344141+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:05.344141+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9681492,"num_objects":50,"num_object_clones":44,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":74213,"num_write":256,"num_write_kb":71774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":50}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"393'88","reported_seq":524,"reported_epoch":399,"state":"active+clean","last_fresh":"2023-03-29T14:09:05.337809+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:05.337809+0000","last_peered":"2023-03-29T14:09:05.337809+0000","last_clean":"2023-03-29T14:09:05.337809+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:05.337809+0000","last_undegraded":"2023-03-29T14:09:05.337809+0000","last_fullsized":"2023-03-29T14:09:05.337809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5882780,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":113,"num_read_kb":57968,"num_write":162,"num_write_kb":45532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":313,"reported_epoch":403,"state":"active+undersized","last_fresh":"2023-03-29T14:09:09.465225+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:09.465225+0000","last_peered":"2023-03-29T14:09:09.465225+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:09.465225+0000","last_undegraded":"2023-03-29T14:09:09.465225+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"390'98","reported_seq":303,"reported_epoch":403,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:08.408669+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:08.408669+0000","last_peered":"2023-03-29T14:09:08.408669+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:08.408669+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084860,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":18402,"num_write":120,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":340,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-03-29T14:09:05.329018+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:05.329018+0000","last_peered":"2023-03-29T14:09:05.329018+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:05.329018+0000","last_undegraded":"2023-03-29T14:09:05.329018+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"396'98","reported_seq":546,"reported_epoch":403,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:08.397704+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:08.397704+0000","last_peered":"2023-03-29T14:09:08.397704+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:08.397704+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6286148,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":107603,"num_write":190,"num_write_kb":55842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":174,"reported_epoch":403,"state":"active+undersized","last_fresh":"2023-03-29T14:09:09.429372+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:09.429372+0000","last_peered":"2023-03-29T14:09:09.429372+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:09.429372+0000","last_undegraded":"2023-03-29T14:09:09.429372+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"393'140","reported_seq":631,"reported_epoch":403,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:09.474429+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:09.474429+0000","last_peered":"2023-03-29T14:09:09.474429+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:09.474429+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10824302,"num_objects":52,"num_object_clones":46,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":219,"num_read_kb":130738,"num_write":268,"num_write_kb":79606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":52}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":340,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-03-29T14:09:05.334586+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:05.334586+0000","last_peered":"2023-03-29T14:09:05.334586+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:05.334586+0000","last_undegraded":"2023-03-29T14:09:05.334586+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":538,"reported_epoch":403,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:09.457268+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:09.457268+0000","last_peered":"2023-03-29T14:09:09.457268+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:09.457268+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"393'198","reported_seq":820,"reported_epoch":399,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:05.338872+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:05.338872+0000","last_peered":"2023-03-29T14:09:05.338872+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:05.338872+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19900412,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":354,"num_read_kb":239220,"num_write":384,"num_write_kb":111100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":72}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":340,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-03-29T14:09:05.336642+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:05.336642+0000","last_peered":"2023-03-29T14:09:05.336642+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:05.336642+0000","last_undegraded":"2023-03-29T14:09:05.336642+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"403'332","reported_seq":1076,"reported_epoch":403,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:10.484691+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:10.484691+0000","last_peered":"2023-03-29T14:09:10.484691+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:10.484691+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28875944,"num_objects":118,"num_object_clones":104,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":521,"num_read_kb":319663,"num_write":622,"num_write_kb":180308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":313,"reported_epoch":403,"state":"active+undersized","last_fresh":"2023-03-29T14:09:09.412560+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:09.412560+0000","last_peered":"2023-03-29T14:09:09.412560+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:09.412560+0000","last_undegraded":"2023-03-29T14:09:09.412560+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":568,"reported_epoch":403,"state":"active+clean","last_fresh":"2023-03-29T14:09:09.635387+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:09.635387+0000","last_peered":"2023-03-29T14:09:09.635387+0000","last_clean":"2023-03-29T14:09:09.635387+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:09.635387+0000","last_undegraded":"2023-03-29T14:09:09.635387+0000","last_fullsized":"2023-03-29T14:09:09.635387+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":145,"num_read_kb":79798,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"384'225","reported_seq":787,"reported_epoch":403,"state":"active+clean","last_fresh":"2023-03-29T14:09:09.447390+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:09.447390+0000","last_peered":"2023-03-29T14:09:09.447390+0000","last_clean":"2023-03-29T14:09:09.447390+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:09.447390+0000","last_undegraded":"2023-03-29T14:09:09.447390+0000","last_fullsized":"2023-03-29T14:09:09.447390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14219712,"num_objects":80,"num_object_clones":70,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":279,"num_read_kb":166588,"num_write":412,"num_write_kb":119550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":313,"reported_epoch":403,"state":"active+undersized","last_fresh":"2023-03-29T14:09:09.433470+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:09.433470+0000","last_peered":"2023-03-29T14:09:09.433470+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:09.433470+0000","last_undegraded":"2023-03-29T14:09:09.433470+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":340,"reported_epoch":399,"state":"active+undersized","last_fresh":"2023-03-29T14:09:05.341206+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:05.341206+0000","last_peered":"2023-03-29T14:09:05.341206+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:05.341206+0000","last_undegraded":"2023-03-29T14:09:05.341206+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"395'221","reported_seq":811,"reported_epoch":403,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:08.416526+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:08.416526+0000","last_peered":"2023-03-29T14:09:08.416526+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:08.416526+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18959154,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":312,"num_read_kb":193896,"num_write":420,"num_write_kb":124398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":80}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":223679766,"num_objects":902,"num_object_clones":810,"num_object_copies":2706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3497,"num_read_kb":2128523,"num_write":4756,"num_write_kb":1379152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2560,"ondisk_log_size":2560,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":509132,"kb_used_data":486276,"kb_used_omap":0,"kb_used_meta":22848,"kb_avail":93862708,"statfs":{"total":96636764160,"available":96115412992,"internally_reserved":0,"allocated":497946624,"data_stored":617161986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23396352},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706088,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":568864,"kb_used_data":544664,"kb_used_omap":0,"kb_used_meta":24192,"kb_avail":93802976,"statfs":{"total":96636764160,"available":96054247424,"internally_reserved":0,"allocated":557735936,"data_stored":691786708,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24772608},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706088,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":516384,"kb_used_data":493080,"kb_used_omap":0,"kb_used_meta":23296,"kb_avail":93855456,"statfs":{"total":96636764160,"available":96107986944,"internally_reserved":0,"allocated":504913920,"data_stored":626434380,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23855104},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:11.685 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.f 3.5 1.0 3.2 3.6 2023-03-29T14:09:11.809 INFO:tasks.rados.rados.0.smithi017.stdout:1639: finishing write tid 1 to smithi017107884-38 2023-03-29T14:09:11.809 INFO:tasks.rados.rados.0.smithi017.stdout:1638: finishing write tid 3 to smithi017107884-3 2023-03-29T14:09:11.809 INFO:tasks.rados.rados.0.smithi017.stdout:1638: finishing write tid 4 to smithi017107884-3 2023-03-29T14:09:11.809 INFO:tasks.rados.rados.0.smithi017.stdout:1638: finishing write tid 5 to smithi017107884-3 2023-03-29T14:09:11.809 INFO:tasks.rados.rados.0.smithi017.stdout:1638: oid 3 updating version 0 to 337 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1638: oid 3 version 337 is already newer than 337 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1638: oid 3 version 337 is already newer than 337 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1638: oid 3 version 337 is already newer than 337 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 337 (ObjNum 790 snap 191 seq_num 790) dirty exists 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1638: left oid 3 (ObjNum 790 snap 191 seq_num 790) 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1639: finishing write tid 2 to smithi017107884-38 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1640: finishing write tid 1 to smithi017107884-8 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1639: finishing write tid 3 to smithi017107884-38 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1639: finishing write tid 4 to smithi017107884-38 2023-03-29T14:09:11.810 INFO:tasks.rados.rados.0.smithi017.stdout:1639: finishing write tid 5 to smithi017107884-38 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1639: oid 38 updating version 0 to 50 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1639: oid 38 updating version 50 to 54 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1639: oid 38 version 54 is already newer than 54 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1639: oid 38 version 54 is already newer than 54 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 54 (ObjNum 791 snap 191 seq_num 791) dirty exists 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1639: left oid 38 (ObjNum 791 snap 191 seq_num 791) 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1640: finishing write tid 2 to smithi017107884-8 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1640: finishing write tid 3 to smithi017107884-8 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1640: finishing write tid 4 to smithi017107884-8 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1640: finishing write tid 5 to smithi017107884-8 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1640: oid 8 updating version 0 to 223 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1640: oid 8 updating version 223 to 227 2023-03-29T14:09:11.811 INFO:tasks.rados.rados.0.smithi017.stdout:1640: oid 8 version 227 is already newer than 227 2023-03-29T14:09:11.812 INFO:tasks.rados.rados.0.smithi017.stdout:1640: oid 8 version 227 is already newer than 227 2023-03-29T14:09:11.812 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 227 (ObjNum 792 snap 191 seq_num 792) dirty exists 2023-03-29T14:09:11.812 INFO:tasks.rados.rados.0.smithi017.stdout:1640: left oid 8 (ObjNum 792 snap 191 seq_num 792) 2023-03-29T14:09:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:11.849+0000 7f59180bd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:11.861+0000 7f59180bd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:12.069 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.5,3.6] on osd.0 to force-backfill; instructing pg(s) [1.0] on osd.1 to force-backfill; instructing pg(s) [3.2,3.f] on osd.3 to force-backfill; 2023-03-29T14:09:12.069 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:09:12.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~242s 2023-03-29T14:09:12.350 INFO:tasks.rados.rados.0.smithi017.stdout:1633: done (7 left) 2023-03-29T14:09:12.350 INFO:tasks.rados.rados.0.smithi017.stdout:1635: done (6 left) 2023-03-29T14:09:12.350 INFO:tasks.rados.rados.0.smithi017.stdout:1637: done (5 left) 2023-03-29T14:09:12.350 INFO:tasks.rados.rados.0.smithi017.stdout:1638: done (4 left) 2023-03-29T14:09:12.350 INFO:tasks.rados.rados.0.smithi017.stdout:1639: done (3 left) 2023-03-29T14:09:12.351 INFO:tasks.rados.rados.0.smithi017.stdout:1640: done (2 left) 2023-03-29T14:09:12.351 INFO:tasks.rados.rados.0.smithi017.stdout:1641: done (1 left) 2023-03-29T14:09:12.351 INFO:tasks.rados.rados.0.smithi017.stdout:1642: done (0 left) 2023-03-29T14:09:12.351 INFO:tasks.rados.rados.0.smithi017.stdout:1643: read oid 26 snap -1 2023-03-29T14:09:12.351 INFO:tasks.rados.rados.0.smithi017.stdout:1643: expect (ObjNum 775 snap 188 seq_num 775) 2023-03-29T14:09:12.351 INFO:tasks.rados.rados.0.smithi017.stdout:1644: snap_create 2023-03-29T14:09:12.397 INFO:tasks.rados.rados.0.smithi017.stdout:1643: expect (ObjNum 775 snap 188 seq_num 775) 2023-03-29T14:09:13.117 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:13.247 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:13.244+0000 7fb2d1e95700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:13.250 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:13.249+0000 7fb2d1e95700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:13.260 INFO:tasks.rados.rados.0.smithi017.stdout:1643: done (1 left) 2023-03-29T14:09:13.260 INFO:tasks.rados.rados.0.smithi017.stdout:1644: done (0 left) 2023-03-29T14:09:13.260 INFO:tasks.rados.rados.0.smithi017.stdout:1645: read oid 19 snap -1 2023-03-29T14:09:13.260 INFO:tasks.rados.rados.0.smithi017.stdout:1645: expect (ObjNum 777 snap 189 seq_num 777) 2023-03-29T14:09:13.260 INFO:tasks.rados.rados.0.smithi017.stdout:1646: read oid 36 snap -1 2023-03-29T14:09:13.260 INFO:tasks.rados.rados.0.smithi017.stdout:1646: expect deleted 2023-03-29T14:09:13.261 INFO:tasks.rados.rados.0.smithi017.stdout:1647: write (excl) oid 25 current snap is 193 2023-03-29T14:09:13.261 INFO:tasks.rados.rados.0.smithi017.stdout:1647: seq_num 793 ranges {527951=573169,1504456=556933,2743845=467765} 2023-03-29T14:09:13.266 INFO:tasks.rados.rados.0.smithi017.stdout:1647: writing smithi017107884-25 from 527951 to 1101120 tid 1 2023-03-29T14:09:13.271 INFO:tasks.rados.rados.0.smithi017.stdout:1647: writing smithi017107884-25 from 1504456 to 2061389 tid 2 2023-03-29T14:09:13.278 INFO:tasks.rados.rados.0.smithi017.stdout:1647: writing smithi017107884-25 from 2743845 to 3211610 tid 3 2023-03-29T14:09:13.278 INFO:tasks.rados.rados.0.smithi017.stdout:1648: read oid 45 snap -1 2023-03-29T14:09:13.278 INFO:tasks.rados.rados.0.smithi017.stdout:1648: expect (ObjNum 767 snap 188 seq_num 767) 2023-03-29T14:09:13.278 INFO:tasks.rados.rados.0.smithi017.stdout:1649: snap_create 2023-03-29T14:09:13.387 INFO:tasks.rados.rados.0.smithi017.stdout:1645: expect (ObjNum 777 snap 189 seq_num 777) 2023-03-29T14:09:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:13.514 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:13.515 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:13.630 INFO:tasks.rados.rados.0.smithi017.stdout:1647: finishing write tid 1 to smithi017107884-25 2023-03-29T14:09:13.630 INFO:tasks.rados.rados.0.smithi017.stdout:1648: expect (ObjNum 767 snap 188 seq_num 767) 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: finishing write tid 2 to smithi017107884-25 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: finishing write tid 3 to smithi017107884-25 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: finishing write tid 4 to smithi017107884-25 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: finishing write tid 5 to smithi017107884-25 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: oid 25 updating version 0 to 273 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: oid 25 version 273 is already newer than 269 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: oid 25 version 273 is already newer than 273 2023-03-29T14:09:13.748 INFO:tasks.rados.rados.0.smithi017.stdout:1647: oid 25 version 273 is already newer than 273 2023-03-29T14:09:13.749 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 273 (ObjNum 793 snap 193 seq_num 793) dirty exists 2023-03-29T14:09:13.749 INFO:tasks.rados.rados.0.smithi017.stdout:1647: left oid 25 (ObjNum 793 snap 193 seq_num 793) 2023-03-29T14:09:14.261 INFO:tasks.rados.rados.0.smithi017.stdout:1645: done (4 left) 2023-03-29T14:09:14.261 INFO:tasks.rados.rados.0.smithi017.stdout:1646: done (3 left) 2023-03-29T14:09:14.261 INFO:tasks.rados.rados.0.smithi017.stdout:1647: done (2 left) 2023-03-29T14:09:14.262 INFO:tasks.rados.rados.0.smithi017.stdout:1648: done (1 left) 2023-03-29T14:09:14.262 INFO:tasks.rados.rados.0.smithi017.stdout:1649: done (0 left) 2023-03-29T14:09:14.262 INFO:tasks.rados.rados.0.smithi017.stdout:1650: read oid 33 snap -1 2023-03-29T14:09:14.262 INFO:tasks.rados.rados.0.smithi017.stdout:1650: expect deleted 2023-03-29T14:09:14.262 INFO:tasks.rados.rados.0.smithi017.stdout:1651: read oid 43 snap 191 2023-03-29T14:09:14.262 INFO:tasks.rados.rados.0.smithi017.stdout:1651: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:14.262 INFO:tasks.rados.rados.0.smithi017.stdout:1652: delete oid 38 current snap is 194 2023-03-29T14:09:14.382 INFO:tasks.rados.rados.0.smithi017.stdout:1651: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:14.509 INFO:tasks.rados.rados.0.smithi017.stdout:1650: done (2 left) 2023-03-29T14:09:14.509 INFO:tasks.rados.rados.0.smithi017.stdout:1651: done (1 left) 2023-03-29T14:09:14.509 INFO:tasks.rados.rados.0.smithi017.stdout:1652: done (0 left) 2023-03-29T14:09:14.510 INFO:tasks.rados.rados.0.smithi017.stdout:1653: read oid 27 snap 191 2023-03-29T14:09:14.510 INFO:tasks.rados.rados.0.smithi017.stdout:1653: expect deleted 2023-03-29T14:09:14.510 INFO:tasks.rados.rados.0.smithi017.stdout:1654: write (excl) oid 36 current snap is 194 2023-03-29T14:09:14.510 INFO:tasks.rados.rados.0.smithi017.stdout:1654: seq_num 795 ranges {561565=723667,1905067=560602,3082338=279227} 2023-03-29T14:09:14.516 INFO:tasks.rados.rados.0.smithi017.stdout:1654: writing smithi017107884-36 from 561565 to 1285232 tid 1 2023-03-29T14:09:14.523 INFO:tasks.rados.rados.0.smithi017.stdout:1654: writing smithi017107884-36 from 1905067 to 2465669 tid 2 2023-03-29T14:09:14.528 INFO:tasks.rados.rados.0.smithi017.stdout:1654: writing smithi017107884-36 from 3082338 to 3361565 tid 3 2023-03-29T14:09:14.528 INFO:tasks.rados.rados.0.smithi017.stdout:1655: snap_remove snap 188 2023-03-29T14:09:15.352 INFO:tasks.rados.rados.0.smithi017.stdout:1655: done (2 left) 2023-03-29T14:09:15.352 INFO:tasks.rados.rados.0.smithi017.stdout:1656: delete oid 50 current snap is 194 2023-03-29T14:09:15.352 INFO:tasks.rados.rados.0.smithi017.stdout:1654: finishing write tid 1 to smithi017107884-36 2023-03-29T14:09:15.352 INFO:tasks.rados.rados.0.smithi017.stdout:1654: finishing write tid 2 to smithi017107884-36 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: finishing write tid 3 to smithi017107884-36 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: finishing write tid 4 to smithi017107884-36 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: finishing write tid 5 to smithi017107884-36 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: oid 36 updating version 0 to 144 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: oid 36 version 144 is already newer than 141 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: oid 36 version 144 is already newer than 144 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: oid 36 version 144 is already newer than 144 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 144 (ObjNum 795 snap 194 seq_num 795) dirty exists 2023-03-29T14:09:15.353 INFO:tasks.rados.rados.0.smithi017.stdout:1654: left oid 36 (ObjNum 795 snap 194 seq_num 795) 2023-03-29T14:09:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1653: done (2 left) 2023-03-29T14:09:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1654: done (1 left) 2023-03-29T14:09:15.475 INFO:tasks.rados.rados.0.smithi017.stdout:1656: done (0 left) 2023-03-29T14:09:15.476 INFO:tasks.rados.rados.0.smithi017.stdout:1657: write oid 23 current snap is 194 2023-03-29T14:09:15.476 INFO:tasks.rados.rados.0.smithi017.stdout:1657: seq_num 797 ranges {595179=474165,1622031=564271,2737184=629206} 2023-03-29T14:09:15.481 INFO:tasks.rados.rados.0.smithi017.stdout:1657: writing smithi017107884-23 from 595179 to 1069344 tid 1 2023-03-29T14:09:15.487 INFO:tasks.rados.rados.0.smithi017.stdout:1657: writing smithi017107884-23 from 1622031 to 2186302 tid 2 2023-03-29T14:09:15.494 INFO:tasks.rados.rados.0.smithi017.stdout:1657: writing smithi017107884-23 from 2737184 to 3366390 tid 3 2023-03-29T14:09:15.494 INFO:tasks.rados.rados.0.smithi017.stdout:1658: read oid 32 snap -1 2023-03-29T14:09:15.494 INFO:tasks.rados.rados.0.smithi017.stdout:1658: expect (ObjNum 667 snap 163 seq_num 667) 2023-03-29T14:09:15.494 INFO:tasks.rados.rados.0.smithi017.stdout:1659: snap_create 2023-03-29T14:09:15.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:15.524 INFO:tasks.rados.rados.0.smithi017.stdout:1657: finishing write tid 1 to smithi017107884-23 2023-03-29T14:09:15.548 INFO:tasks.rados.rados.0.smithi017.stdout:1657: finishing write tid 2 to smithi017107884-23 2023-03-29T14:09:15.552 INFO:tasks.rados.rados.0.smithi017.stdout:1657: finishing write tid 3 to smithi017107884-23 2023-03-29T14:09:15.556 INFO:tasks.rados.rados.0.smithi017.stdout:1657: finishing write tid 4 to smithi017107884-23 2023-03-29T14:09:15.560 INFO:tasks.rados.rados.0.smithi017.stdout:1657: finishing write tid 5 to smithi017107884-23 2023-03-29T14:09:15.560 INFO:tasks.rados.rados.0.smithi017.stdout:1657: oid 23 updating version 0 to 342 2023-03-29T14:09:15.560 INFO:tasks.rados.rados.0.smithi017.stdout:1657: oid 23 version 342 is already newer than 340 2023-03-29T14:09:15.560 INFO:tasks.rados.rados.0.smithi017.stdout:1657: oid 23 version 342 is already newer than 342 2023-03-29T14:09:15.560 INFO:tasks.rados.rados.0.smithi017.stdout:1657: oid 23 version 342 is already newer than 341 2023-03-29T14:09:15.561 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 342 (ObjNum 797 snap 194 seq_num 797) dirty exists 2023-03-29T14:09:15.561 INFO:tasks.rados.rados.0.smithi017.stdout:1657: left oid 23 (ObjNum 797 snap 194 seq_num 797) 2023-03-29T14:09:15.586 INFO:tasks.rados.rados.0.smithi017.stdout:1658: expect (ObjNum 667 snap 163 seq_num 667) 2023-03-29T14:09:15.649 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:15.646+0000 7f99e31e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:15.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:15.651+0000 7f99e31e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:15.714 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:15.717 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:15.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:16.356 INFO:tasks.rados.rados.0.smithi017.stdout:1657: done (2 left) 2023-03-29T14:09:16.356 INFO:tasks.rados.rados.0.smithi017.stdout:1658: done (1 left) 2023-03-29T14:09:16.356 INFO:tasks.rados.rados.0.smithi017.stdout:1659: done (0 left) 2023-03-29T14:09:16.356 INFO:tasks.rados.rados.0.smithi017.stdout:1660: read oid 1 snap -1 2023-03-29T14:09:16.356 INFO:tasks.rados.rados.0.smithi017.stdout:1660: expect (ObjNum 745 snap 181 seq_num 745) 2023-03-29T14:09:16.356 INFO:tasks.rados.rados.0.smithi017.stdout:1661: write (excl) oid 31 current snap is 195 2023-03-29T14:09:16.356 INFO:tasks.rados.rados.0.smithi017.stdout:1661: seq_num 798 ranges {611986=549414,1564160=424282,2764607=440478} 2023-03-29T14:09:16.362 INFO:tasks.rados.rados.0.smithi017.stdout:1661: writing smithi017107884-31 from 611986 to 1161400 tid 1 2023-03-29T14:09:16.366 INFO:tasks.rados.rados.0.smithi017.stdout:1661: writing smithi017107884-31 from 1564160 to 1988442 tid 2 2023-03-29T14:09:16.373 INFO:tasks.rados.rados.0.smithi017.stdout:1661: writing smithi017107884-31 from 2764607 to 3205085 tid 3 2023-03-29T14:09:16.373 INFO:tasks.rados.rados.0.smithi017.stdout:1662: write oid 10 current snap is 195 2023-03-29T14:09:16.373 INFO:tasks.rados.rados.0.smithi017.stdout:1662: seq_num 799 ranges {628793=741016,2138995=567940,3192030=236763} 2023-03-29T14:09:16.380 INFO:tasks.rados.rados.0.smithi017.stdout:1662: writing smithi017107884-10 from 628793 to 1369809 tid 1 2023-03-29T14:09:16.388 INFO:tasks.rados.rados.0.smithi017.stdout:1662: writing smithi017107884-10 from 2138995 to 2706935 tid 2 2023-03-29T14:09:16.394 INFO:tasks.rados.rados.0.smithi017.stdout:1662: writing smithi017107884-10 from 3192030 to 3428793 tid 3 2023-03-29T14:09:16.394 INFO:tasks.rados.rados.0.smithi017.stdout:1663: write oid 50 current snap is 195 2023-03-29T14:09:16.394 INFO:tasks.rados.rados.0.smithi017.stdout:1663: seq_num 800 ranges {645600=416265,1797477=711598,3219453=226147} 2023-03-29T14:09:16.398 INFO:tasks.rados.rados.0.smithi017.stdout:1663: writing smithi017107884-50 from 645600 to 1061865 tid 1 2023-03-29T14:09:16.406 INFO:tasks.rados.rados.0.smithi017.stdout:1663: writing smithi017107884-50 from 1797477 to 2509075 tid 2 2023-03-29T14:09:16.411 INFO:tasks.rados.rados.0.smithi017.stdout:1663: writing smithi017107884-50 from 3219453 to 3445600 tid 3 2023-03-29T14:09:16.412 INFO:tasks.rados.rados.0.smithi017.stdout:1664: write oid 38 current snap is 195 2023-03-29T14:09:16.412 INFO:tasks.rados.rados.0.smithi017.stdout:1664: seq_num 801 ranges {662407=491514,1855959=571609,2846876=615531} 2023-03-29T14:09:16.417 INFO:tasks.rados.rados.0.smithi017.stdout:1664: writing smithi017107884-38 from 662407 to 1153921 tid 1 2023-03-29T14:09:16.425 INFO:tasks.rados.rados.0.smithi017.stdout:1664: writing smithi017107884-38 from 1855959 to 2427568 tid 2 2023-03-29T14:09:16.431 INFO:tasks.rados.rados.0.smithi017.stdout:1664: writing smithi017107884-38 from 2846876 to 3462407 tid 3 2023-03-29T14:09:16.431 INFO:tasks.rados.rados.0.smithi017.stdout:1665: read oid 37 snap 191 2023-03-29T14:09:16.431 INFO:tasks.rados.rados.0.smithi017.stdout:1665: expect (ObjNum 761 snap 186 seq_num 761) 2023-03-29T14:09:16.431 INFO:tasks.rados.rados.0.smithi017.stdout:1666: write oid 28 current snap is 195 2023-03-29T14:09:16.431 INFO:tasks.rados.rados.0.smithi017.stdout:1666: seq_num 802 ranges {679214=566763,1798088=715267,3157946=321268} 2023-03-29T14:09:16.437 INFO:tasks.rados.rados.0.smithi017.stdout:1666: writing smithi017107884-28 from 679214 to 1245977 tid 1 2023-03-29T14:09:16.444 INFO:tasks.rados.rados.0.smithi017.stdout:1666: writing smithi017107884-28 from 1798088 to 2513355 tid 2 2023-03-29T14:09:16.450 INFO:tasks.rados.rados.0.smithi017.stdout:1666: writing smithi017107884-28 from 3157946 to 3479214 tid 3 2023-03-29T14:09:16.450 INFO:tasks.rados.rados.0.smithi017.stdout:1667: read oid 5 snap -1 2023-03-29T14:09:16.450 INFO:tasks.rados.rados.0.smithi017.stdout:1667: expect (ObjNum 721 snap 177 seq_num 721) 2023-03-29T14:09:16.450 INFO:tasks.rados.rados.0.smithi017.stdout:1668: read oid 13 snap -1 2023-03-29T14:09:16.450 INFO:tasks.rados.rados.0.smithi017.stdout:1668: expect deleted 2023-03-29T14:09:16.450 INFO:tasks.rados.rados.0.smithi017.stdout:1669: snap_create 2023-03-29T14:09:16.497 INFO:tasks.rados.rados.0.smithi017.stdout:1660: expect (ObjNum 745 snap 181 seq_num 745) 2023-03-29T14:09:16.601 INFO:tasks.rados.rados.0.smithi017.stdout:1661: finishing write tid 1 to smithi017107884-31 2023-03-29T14:09:16.601 INFO:tasks.rados.rados.0.smithi017.stdout:1665: expect (ObjNum 761 snap 186 seq_num 761) 2023-03-29T14:09:16.749 INFO:tasks.rados.rados.0.smithi017.stdout:1663: finishing write tid 1 to smithi017107884-50 2023-03-29T14:09:16.750 INFO:tasks.rados.rados.0.smithi017.stdout:1663: finishing write tid 2 to smithi017107884-50 2023-03-29T14:09:16.750 INFO:tasks.rados.rados.0.smithi017.stdout:1662: finishing write tid 1 to smithi017107884-10 2023-03-29T14:09:16.750 INFO:tasks.rados.rados.0.smithi017.stdout:1662: finishing write tid 2 to smithi017107884-10 2023-03-29T14:09:16.750 INFO:tasks.rados.rados.0.smithi017.stdout:1662: finishing write tid 3 to smithi017107884-10 2023-03-29T14:09:16.750 INFO:tasks.rados.rados.0.smithi017.stdout:1663: finishing write tid 3 to smithi017107884-50 2023-03-29T14:09:16.750 INFO:tasks.rados.rados.0.smithi017.stdout:1662: finishing write tid 4 to smithi017107884-10 2023-03-29T14:09:16.750 INFO:tasks.rados.rados.0.smithi017.stdout:1662: finishing write tid 5 to smithi017107884-10 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1662: oid 10 updating version 0 to 255 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1662: oid 10 version 255 is already newer than 255 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1662: oid 10 version 255 is already newer than 255 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1662: oid 10 version 255 is already newer than 255 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 10 v 255 (ObjNum 799 snap 195 seq_num 799) dirty exists 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1662: left oid 10 (ObjNum 799 snap 195 seq_num 799) 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1666: finishing write tid 1 to smithi017107884-28 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1663: finishing write tid 4 to smithi017107884-50 2023-03-29T14:09:16.751 INFO:tasks.rados.rados.0.smithi017.stdout:1666: finishing write tid 2 to smithi017107884-28 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:1663: finishing write tid 5 to smithi017107884-50 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:1663: oid 50 updating version 0 to 150 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:1663: oid 50 version 150 is already newer than 150 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:1663: oid 50 version 150 is already newer than 150 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:1663: oid 50 version 150 is already newer than 150 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 50 v 150 (ObjNum 800 snap 195 seq_num 800) dirty exists 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:1663: left oid 50 (ObjNum 800 snap 195 seq_num 800) 2023-03-29T14:09:16.752 INFO:tasks.rados.rados.0.smithi017.stdout:1664: finishing write tid 1 to smithi017107884-38 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1664: finishing write tid 2 to smithi017107884-38 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1664: finishing write tid 3 to smithi017107884-38 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1666: finishing write tid 3 to smithi017107884-28 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1666: finishing write tid 4 to smithi017107884-28 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1666: finishing write tid 5 to smithi017107884-28 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1666: oid 28 updating version 0 to 93 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1666: oid 28 version 93 is already newer than 93 2023-03-29T14:09:16.753 INFO:tasks.rados.rados.0.smithi017.stdout:1666: oid 28 version 93 is already newer than 93 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1666: oid 28 version 93 is already newer than 93 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 93 (ObjNum 802 snap 195 seq_num 802) dirty exists 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1666: left oid 28 (ObjNum 802 snap 195 seq_num 802) 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1664: finishing write tid 4 to smithi017107884-38 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1661: finishing write tid 2 to smithi017107884-31 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1664: finishing write tid 5 to smithi017107884-38 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1664: oid 38 updating version 0 to 60 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1664: oid 38 version 60 is already newer than 59 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1664: oid 38 version 60 is already newer than 60 2023-03-29T14:09:16.754 INFO:tasks.rados.rados.0.smithi017.stdout:1664: oid 38 version 60 is already newer than 60 2023-03-29T14:09:16.755 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 38 v 60 (ObjNum 801 snap 195 seq_num 801) dirty exists 2023-03-29T14:09:16.755 INFO:tasks.rados.rados.0.smithi017.stdout:1664: left oid 38 (ObjNum 801 snap 195 seq_num 801) 2023-03-29T14:09:16.755 INFO:tasks.rados.rados.0.smithi017.stdout:1667: expect (ObjNum 721 snap 177 seq_num 721) 2023-03-29T14:09:16.944 INFO:tasks.rados.rados.0.smithi017.stdout:1661: finishing write tid 3 to smithi017107884-31 2023-03-29T14:09:16.944 INFO:tasks.rados.rados.0.smithi017.stdout:1661: finishing write tid 4 to smithi017107884-31 2023-03-29T14:09:16.944 INFO:tasks.rados.rados.0.smithi017.stdout:1661: finishing write tid 5 to smithi017107884-31 2023-03-29T14:09:16.944 INFO:tasks.rados.rados.0.smithi017.stdout:1661: oid 31 updating version 0 to 147 2023-03-29T14:09:16.945 INFO:tasks.rados.rados.0.smithi017.stdout:1661: oid 31 version 147 is already newer than 143 2023-03-29T14:09:16.945 INFO:tasks.rados.rados.0.smithi017.stdout:1661: oid 31 version 147 is already newer than 147 2023-03-29T14:09:16.945 INFO:tasks.rados.rados.0.smithi017.stdout:1661: oid 31 version 147 is already newer than 147 2023-03-29T14:09:16.945 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 147 (ObjNum 798 snap 195 seq_num 798) dirty exists 2023-03-29T14:09:16.945 INFO:tasks.rados.rados.0.smithi017.stdout:1661: left oid 31 (ObjNum 798 snap 195 seq_num 798) 2023-03-29T14:09:17.082 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:17.082 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:17.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:17.217 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:17.212+0000 7fea0dce8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:17.221 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:17.220+0000 7fea0dce8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:17.359 INFO:tasks.rados.rados.0.smithi017.stdout:1660: done (9 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1661: done (8 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1662: done (7 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1663: done (6 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1664: done (5 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1665: done (4 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1666: done (3 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1667: done (2 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1668: done (1 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1669: done (0 left) 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1670: write (excl) oid 13 current snap is 196 2023-03-29T14:09:17.360 INFO:tasks.rados.rados.0.smithi017.stdout:1670: seq_num 803 ranges {696021=642012,1856570=575278,3185369=310652} 2023-03-29T14:09:17.367 INFO:tasks.rados.rados.0.smithi017.stdout:1670: writing smithi017107884-13 from 696021 to 1338033 tid 1 2023-03-29T14:09:17.373 INFO:tasks.rados.rados.0.smithi017.stdout:1670: writing smithi017107884-13 from 1856570 to 2431848 tid 2 2023-03-29T14:09:17.379 INFO:tasks.rados.rados.0.smithi017.stdout:1670: writing smithi017107884-13 from 3185369 to 3496021 tid 3 2023-03-29T14:09:17.379 INFO:tasks.rados.rados.0.smithi017.stdout:1671: write oid 20 current snap is 196 2023-03-29T14:09:17.379 INFO:tasks.rados.rados.0.smithi017.stdout:1671: seq_num 804 ranges {712828=717261,1915052=718936,3212792=300036} 2023-03-29T14:09:17.386 INFO:tasks.rados.rados.0.smithi017.stdout:1671: writing smithi017107884-20 from 712828 to 1430089 tid 1 2023-03-29T14:09:17.393 INFO:tasks.rados.rados.0.smithi017.stdout:1671: writing smithi017107884-20 from 1915052 to 2633988 tid 2 2023-03-29T14:09:17.398 INFO:tasks.rados.rados.0.smithi017.stdout:1671: writing smithi017107884-20 from 3212792 to 3512828 tid 3 2023-03-29T14:09:17.398 INFO:tasks.rados.rados.0.smithi017.stdout:1672: write (excl) oid 18 current snap is 196 2023-03-29T14:09:17.398 INFO:tasks.rados.rados.0.smithi017.stdout:1672: seq_num 805 ranges {729635=792510,1973534=578947,3240215=289420} 2023-03-29T14:09:17.406 INFO:tasks.rados.rados.0.smithi017.stdout:1672: writing smithi017107884-18 from 729635 to 1522145 tid 1 2023-03-29T14:09:17.412 INFO:tasks.rados.rados.0.smithi017.stdout:1672: writing smithi017107884-18 from 1973534 to 2552481 tid 2 2023-03-29T14:09:17.418 INFO:tasks.rados.rados.0.smithi017.stdout:1672: writing smithi017107884-18 from 3240215 to 3529635 tid 3 2023-03-29T14:09:17.418 INFO:tasks.rados.rados.0.smithi017.stdout:1673: write oid 36 current snap is 196 2023-03-29T14:09:17.418 INFO:tasks.rados.rados.0.smithi017.stdout:1673: seq_num 806 ranges {746442=584112,1748369=722605,2983991=562451} 2023-03-29T14:09:17.425 INFO:tasks.rados.rados.0.smithi017.stdout:1673: writing smithi017107884-36 from 746442 to 1330554 tid 1 2023-03-29T14:09:17.426 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:17.429 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:17.431 INFO:tasks.rados.rados.0.smithi017.stdout:1673: writing smithi017107884-36 from 1748369 to 2470974 tid 2 2023-03-29T14:09:17.437 INFO:tasks.rados.rados.0.smithi017.stdout:1673: writing smithi017107884-36 from 2983991 to 3546442 tid 3 2023-03-29T14:09:17.438 INFO:tasks.rados.rados.0.smithi017.stdout:1674: write oid 7 current snap is 196 2023-03-29T14:09:17.438 INFO:tasks.rados.rados.0.smithi017.stdout:1674: seq_num 807 ranges {763249=659361,2090498=582616,3295061=268188} 2023-03-29T14:09:17.442 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":619,"stamp":"2023-03-29T14:09:17.358363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":236423850,"num_objects":928,"num_object_clones":832,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3719,"num_read_kb":2181771,"num_write":5140,"num_write_kb":1416022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2712,"ondisk_log_size":2712,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1777224,"kb_used_data":1694248,"kb_used_omap":0,"kb_used_meta":82944,"kb_avail":375710136,"statfs":{"total":386547056640,"available":384727179264,"internally_reserved":0,"allocated":1734909952,"data_stored":2145653242,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":84934656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10185174,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":39569,"num_write":90,"num_write_kb":27254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.102370"},"pg_stats":[{"pgid":"3.f","version":"396'124","reported_seq":395,"reported_epoch":409,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:14.277559+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:14.277559+0000","last_peered":"2023-03-29T14:09:14.277559+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:14.277559+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13950404,"num_objects":26,"num_object_clones":26,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":65945,"num_write":162,"num_write_kb":47282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":26}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"407'56","reported_seq":429,"reported_epoch":408,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:14.352425+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:14.352425+0000","last_peered":"2023-03-29T14:09:14.352425+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:14.352425+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":66,"num_read_kb":30741,"num_write":102,"num_write_kb":30382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"409'254","reported_seq":912,"reported_epoch":409,"state":"active+clean","last_fresh":"2023-03-29T14:09:16.437346+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:16.437346+0000","last_peered":"2023-03-29T14:09:16.437346+0000","last_clean":"2023-03-29T14:09:16.437346+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:16.437346+0000","last_undegraded":"2023-03-29T14:09:16.437346+0000","last_fullsized":"2023-03-29T14:09:16.437346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29749908,"num_objects":92,"num_object_clones":82,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":258085,"num_write":494,"num_write_kb":146628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"406'273","reported_seq":938,"reported_epoch":409,"state":"active+clean","last_fresh":"2023-03-29T14:09:14.269576+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:14.269576+0000","last_peered":"2023-03-29T14:09:14.269576+0000","last_clean":"2023-03-29T14:09:14.269576+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:14.269576+0000","last_undegraded":"2023-03-29T14:09:14.269576+0000","last_fullsized":"2023-03-29T14:09:14.269576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346276,"num_objects":108,"num_object_clones":100,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":363,"num_read_kb":212499,"num_write":536,"num_write_kb":151626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"399'217","reported_seq":799,"reported_epoch":408,"state":"active+clean","last_fresh":"2023-03-29T14:09:14.281677+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:14.281677+0000","last_peered":"2023-03-29T14:09:14.281677+0000","last_clean":"2023-03-29T14:09:14.281677+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:14.281677+0000","last_undegraded":"2023-03-29T14:09:14.281677+0000","last_fullsized":"2023-03-29T14:09:14.281677+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22489020,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":185362,"num_write":434,"num_write_kb":127506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":345,"reported_epoch":409,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.273295+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:14.273295+0000","last_peered":"2023-03-29T14:09:14.273295+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:14.273295+0000","last_undegraded":"2023-03-29T14:09:14.273295+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"409'146","reported_seq":648,"reported_epoch":409,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:16.530349+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:16.530349+0000","last_peered":"2023-03-29T14:09:16.530349+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:16.530349+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698332,"num_objects":52,"num_object_clones":46,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":163,"num_read_kb":74213,"num_write":266,"num_write_kb":74540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"409'92","reported_seq":543,"reported_epoch":409,"state":"active+clean","last_fresh":"2023-03-29T14:09:16.460668+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:16.460668+0000","last_peered":"2023-03-29T14:09:16.460668+0000","last_clean":"2023-03-29T14:09:16.460668+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:16.460668+0000","last_undegraded":"2023-03-29T14:09:16.460668+0000","last_fullsized":"2023-03-29T14:09:16.460668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12841208,"num_objects":32,"num_object_clones":28,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":64455,"num_write":172,"num_write_kb":48666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":317,"reported_epoch":408,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.321104+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:14.321104+0000","last_peered":"2023-03-29T14:09:14.321104+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:14.321104+0000","last_undegraded":"2023-03-29T14:09:14.321104+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"390'98","reported_seq":308,"reported_epoch":408,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:14.294523+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:14.294523+0000","last_peered":"2023-03-29T14:09:14.294523+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:14.294523+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084860,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":18402,"num_write":120,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":348,"reported_epoch":409,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.271988+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:14.271988+0000","last_peered":"2023-03-29T14:09:14.271988+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:14.271988+0000","last_undegraded":"2023-03-29T14:09:14.271988+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"396'98","reported_seq":551,"reported_epoch":408,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:14.280875+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:14.280875+0000","last_peered":"2023-03-29T14:09:14.280875+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:14.280875+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6286148,"num_objects":40,"num_object_clones":36,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":161,"num_read_kb":107603,"num_write":190,"num_write_kb":55842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":40}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":408,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.314561+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:14.314561+0000","last_peered":"2023-03-29T14:09:14.314561+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:14.314561+0000","last_undegraded":"2023-03-29T14:09:14.314561+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"407'144","reported_seq":652,"reported_epoch":408,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:15.427663+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:15.427663+0000","last_peered":"2023-03-29T14:09:15.427663+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:15.427663+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530196,"num_objects":54,"num_object_clones":48,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":236,"num_read_kb":142916,"num_write":278,"num_write_kb":82662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":54}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":348,"reported_epoch":409,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.265306+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:14.265306+0000","last_peered":"2023-03-29T14:09:14.265306+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:14.265306+0000","last_undegraded":"2023-03-29T14:09:14.265306+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":542,"reported_epoch":408,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:14.287038+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:14.287038+0000","last_peered":"2023-03-29T14:09:14.287038+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:14.287038+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"393'198","reported_seq":831,"reported_epoch":409,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:14.270172+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:14.270172+0000","last_peered":"2023-03-29T14:09:14.270172+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:14.270172+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19900412,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":361,"num_read_kb":246214,"num_write":384,"num_write_kb":111100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":72}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":348,"reported_epoch":409,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.282107+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:14.282107+0000","last_peered":"2023-03-29T14:09:14.282107+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:14.282107+0000","last_undegraded":"2023-03-29T14:09:14.282107+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"408'342","reported_seq":1102,"reported_epoch":408,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:15.575694+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:15.575694+0000","last_peered":"2023-03-29T14:09:15.575694+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:15.575694+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32943238,"num_objects":122,"num_object_clones":108,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":530,"num_read_kb":327695,"num_write":642,"num_write_kb":186618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":122}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":317,"reported_epoch":408,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.295516+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:14.295516+0000","last_peered":"2023-03-29T14:09:14.295516+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:14.295516+0000","last_undegraded":"2023-03-29T14:09:14.295516+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":572,"reported_epoch":408,"state":"active+clean","last_fresh":"2023-03-29T14:09:14.286374+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:14.286374+0000","last_peered":"2023-03-29T14:09:14.286374+0000","last_clean":"2023-03-29T14:09:14.286374+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:14.286374+0000","last_undegraded":"2023-03-29T14:09:14.286374+0000","last_fullsized":"2023-03-29T14:09:14.286374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":145,"num_read_kb":79798,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"384'225","reported_seq":791,"reported_epoch":408,"state":"active+clean","last_fresh":"2023-03-29T14:09:14.301303+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:14.301303+0000","last_peered":"2023-03-29T14:09:14.301303+0000","last_clean":"2023-03-29T14:09:14.301303+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:14.301303+0000","last_undegraded":"2023-03-29T14:09:14.301303+0000","last_fullsized":"2023-03-29T14:09:14.301303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14219712,"num_objects":80,"num_object_clones":70,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":279,"num_read_kb":166588,"num_write":412,"num_write_kb":119550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":317,"reported_epoch":408,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.329738+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:14.329738+0000","last_peered":"2023-03-29T14:09:14.329738+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:14.329738+0000","last_undegraded":"2023-03-29T14:09:14.329738+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":348,"reported_epoch":409,"state":"active+undersized","last_fresh":"2023-03-29T14:09:14.280631+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:14.280631+0000","last_peered":"2023-03-29T14:09:14.280631+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:14.280631+0000","last_undegraded":"2023-03-29T14:09:14.280631+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"404'227","reported_seq":828,"reported_epoch":408,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:14.324033+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:14.324033+0000","last_peered":"2023-03-29T14:09:14.324033+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:14.324033+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17782796,"num_objects":82,"num_object_clones":74,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":322,"num_read_kb":201135,"num_write":430,"num_write_kb":127464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":82}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":235243114,"num_objects":924,"num_object_clones":832,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3577,"num_read_kb":2181651,"num_write":4866,"num_write_kb":1412234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2615,"ondisk_log_size":2615,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706094,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":520360,"kb_used_data":497312,"kb_used_omap":0,"kb_used_meta":23040,"kb_avail":93851480,"statfs":{"total":96636764160,"available":96103915520,"internally_reserved":0,"allocated":509247488,"data_stored":632728867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23592960},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706093,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":578312,"kb_used_data":553920,"kb_used_omap":0,"kb_used_meta":24384,"kb_avail":93793528,"statfs":{"total":96636764160,"available":96044572672,"internally_reserved":0,"allocated":567214080,"data_stored":704745741,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24969216},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706093,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":522608,"kb_used_data":499176,"kb_used_omap":0,"kb_used_meta":23424,"kb_avail":93849232,"statfs":{"total":96636764160,"available":96101613568,"internally_reserved":0,"allocated":511156224,"data_stored":634946409,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":23986176},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:17.445 INFO:tasks.rados.rados.0.smithi017.stdout:1674: writing smithi017107884-7 from 763249 to 1422610 tid 1 2023-03-29T14:09:17.452 INFO:tasks.rados.rados.0.smithi017.stdout:1674: writing smithi017107884-7 from 2090498 to 2673114 tid 2 2023-03-29T14:09:17.457 INFO:tasks.rados.rados.0.smithi017.stdout:1674: writing smithi017107884-7 from 3295061 to 3563249 tid 3 2023-03-29T14:09:17.458 INFO:tasks.rados.rados.0.smithi017.stdout:1675: read oid 4 snap -1 2023-03-29T14:09:17.458 INFO:tasks.rados.rados.0.smithi017.stdout:1675: expect (ObjNum 610 snap 144 seq_num 610) 2023-03-29T14:09:17.458 INFO:tasks.rados.rados.0.smithi017.stdout:1676: read oid 39 snap -1 2023-03-29T14:09:17.458 INFO:tasks.rados.rados.0.smithi017.stdout:1676: expect (ObjNum 673 snap 164 seq_num 673) 2023-03-29T14:09:17.458 INFO:tasks.rados.rados.0.smithi017.stdout:1677: snap_remove snap 192 2023-03-29T14:09:17.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:17.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~248s 2023-03-29T14:09:17.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:17.850+0000 7fb439a03700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:17.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:17.855+0000 7fb439a03700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:17.918 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:17.921 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:17.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1677: done (7 left) 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1678: read oid 43 snap -1 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1678: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1679: delete oid 47 current snap is 196 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1672: finishing write tid 1 to smithi017107884-18 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1674: finishing write tid 1 to smithi017107884-7 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1670: finishing write tid 1 to smithi017107884-13 2023-03-29T14:09:18.363 INFO:tasks.rados.rados.0.smithi017.stdout:1675: expect (ObjNum 610 snap 144 seq_num 610) 2023-03-29T14:09:18.494 INFO:tasks.rados.rados.0.smithi017.stdout:1676: expect (ObjNum 673 snap 164 seq_num 673) 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1674: finishing write tid 2 to smithi017107884-7 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1674: finishing write tid 3 to smithi017107884-7 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: finishing write tid 2 to smithi017107884-18 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: finishing write tid 3 to smithi017107884-18 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: finishing write tid 4 to smithi017107884-18 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: finishing write tid 5 to smithi017107884-18 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: oid 18 updating version 0 to 151 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: oid 18 version 151 is already newer than 151 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: oid 18 version 151 is already newer than 151 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: oid 18 version 151 is already newer than 148 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 151 (ObjNum 805 snap 196 seq_num 805) dirty exists 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1672: left oid 18 (ObjNum 805 snap 196 seq_num 805) 2023-03-29T14:09:18.709 INFO:tasks.rados.rados.0.smithi017.stdout:1674: finishing write tid 4 to smithi017107884-7 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1674: finishing write tid 5 to smithi017107884-7 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1674: oid 7 updating version 0 to 98 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1674: oid 7 version 98 is already newer than 98 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1674: oid 7 version 98 is already newer than 98 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1674: oid 7 version 98 is already newer than 98 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 7 v 98 (ObjNum 807 snap 196 seq_num 807) dirty exists 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1674: left oid 7 (ObjNum 807 snap 196 seq_num 807) 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: finishing write tid 2 to smithi017107884-13 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1671: finishing write tid 1 to smithi017107884-20 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: finishing write tid 3 to smithi017107884-13 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1673: finishing write tid 1 to smithi017107884-36 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1671: finishing write tid 2 to smithi017107884-20 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1673: finishing write tid 2 to smithi017107884-36 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: finishing write tid 4 to smithi017107884-13 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: finishing write tid 5 to smithi017107884-13 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: oid 13 updating version 0 to 229 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: oid 13 version 229 is already newer than 229 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: oid 13 version 229 is already newer than 226 2023-03-29T14:09:18.710 INFO:tasks.rados.rados.0.smithi017.stdout:1670: oid 13 version 229 is already newer than 229 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 229 (ObjNum 803 snap 196 seq_num 803) dirty exists 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1670: left oid 13 (ObjNum 803 snap 196 seq_num 803) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1673: finishing write tid 3 to smithi017107884-36 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1671: finishing write tid 3 to smithi017107884-20 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1673: finishing write tid 4 to smithi017107884-36 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1670: done (8 left) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1672: done (7 left) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1674: done (6 left) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1675: done (5 left) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1676: done (4 left) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1679: done (3 left) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1680: read oid 46 snap -1 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1680: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1673: finishing write tid 5 to smithi017107884-36 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1673: oid 36 updating version 0 to 155 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1673: oid 36 version 155 is already newer than 155 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1673: oid 36 version 155 is already newer than 155 2023-03-29T14:09:18.711 INFO:tasks.rados.rados.0.smithi017.stdout:1673: oid 36 version 155 is already newer than 155 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 36 v 155 (ObjNum 806 snap 196 seq_num 806) dirty exists 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1673: left oid 36 (ObjNum 806 snap 196 seq_num 806) 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1671: finishing write tid 4 to smithi017107884-20 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1673: done (3 left) 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1681: delete oid 3 current snap is 196 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1671: finishing write tid 5 to smithi017107884-20 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1671: oid 20 updating version 0 to 222 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1671: oid 20 version 222 is already newer than 222 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1671: oid 20 version 222 is already newer than 222 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1671: oid 20 version 222 is already newer than 222 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 222 (ObjNum 804 snap 196 seq_num 804) dirty exists 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1671: left oid 20 (ObjNum 804 snap 196 seq_num 804) 2023-03-29T14:09:18.712 INFO:tasks.rados.rados.0.smithi017.stdout:1678: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:18.838 INFO:tasks.rados.rados.0.smithi017.stdout:1680: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:18.954 INFO:tasks.rados.rados.0.smithi017.stdout:1671: done (3 left) 2023-03-29T14:09:18.954 INFO:tasks.rados.rados.0.smithi017.stdout:1678: done (2 left) 2023-03-29T14:09:18.954 INFO:tasks.rados.rados.0.smithi017.stdout:1680: done (1 left) 2023-03-29T14:09:18.955 INFO:tasks.rados.rados.0.smithi017.stdout:1681: done (0 left) 2023-03-29T14:09:18.955 INFO:tasks.rados.rados.0.smithi017.stdout:1682: read oid 4 snap 193 2023-03-29T14:09:18.955 INFO:tasks.rados.rados.0.smithi017.stdout:1682: expect (ObjNum 610 snap 144 seq_num 610) 2023-03-29T14:09:18.955 INFO:tasks.rados.rados.0.smithi017.stdout:1683: write (excl) oid 28 current snap is 196 2023-03-29T14:09:18.955 INFO:tasks.rados.rados.0.smithi017.stdout:1683: seq_num 810 ranges {413670=485108,1465944=729943,2977330=524801} 2023-03-29T14:09:18.959 INFO:tasks.rados.rados.0.smithi017.stdout:1683: writing smithi017107884-28 from 413670 to 898778 tid 1 2023-03-29T14:09:18.966 INFO:tasks.rados.rados.0.smithi017.stdout:1683: writing smithi017107884-28 from 1465944 to 2195887 tid 2 2023-03-29T14:09:18.973 INFO:tasks.rados.rados.0.smithi017.stdout:1683: writing smithi017107884-28 from 2977330 to 3502131 tid 3 2023-03-29T14:09:18.974 INFO:tasks.rados.rados.0.smithi017.stdout:1684: read oid 34 snap -1 2023-03-29T14:09:18.974 INFO:tasks.rados.rados.0.smithi017.stdout:1684: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:09:18.974 INFO:tasks.rados.rados.0.smithi017.stdout:1685: write (excl) oid 24 current snap is 196 2023-03-29T14:09:18.974 INFO:tasks.rados.rados.0.smithi017.stdout:1685: seq_num 811 ranges {430477=560357,1408073=473601,2488400=736073} 2023-03-29T14:09:18.979 INFO:tasks.rados.rados.0.smithi017.stdout:1685: writing smithi017107884-24 from 430477 to 990834 tid 1 2023-03-29T14:09:18.984 INFO:tasks.rados.rados.0.smithi017.stdout:1685: writing smithi017107884-24 from 1408073 to 1881674 tid 2 2023-03-29T14:09:18.992 INFO:tasks.rados.rados.0.smithi017.stdout:1685: writing smithi017107884-24 from 2488400 to 3224473 tid 3 2023-03-29T14:09:18.992 INFO:tasks.rados.rados.0.smithi017.stdout:1686: write oid 15 current snap is 196 2023-03-29T14:09:18.992 INFO:tasks.rados.rados.0.smithi017.stdout:1686: seq_num 812 ranges {447284=635606,1866555=733612,3315823=331461} 2023-03-29T14:09:18.998 INFO:tasks.rados.rados.0.smithi017.stdout:1686: writing smithi017107884-15 from 447284 to 1082890 tid 1 2023-03-29T14:09:19.007 INFO:tasks.rados.rados.0.smithi017.stdout:1686: writing smithi017107884-15 from 1866555 to 2600167 tid 2 2023-03-29T14:09:19.013 INFO:tasks.rados.rados.0.smithi017.stdout:1686: writing smithi017107884-15 from 3315823 to 3647284 tid 3 2023-03-29T14:09:19.013 INFO:tasks.rados.rados.0.smithi017.stdout:1687: snap_create 2023-03-29T14:09:19.013 INFO:tasks.rados.rados.0.smithi017.stdout:1682: expect (ObjNum 610 snap 144 seq_num 610) 2023-03-29T14:09:19.147 INFO:tasks.rados.rados.0.smithi017.stdout:1684: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: finishing write tid 1 to smithi017107884-28 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1685: finishing write tid 1 to smithi017107884-24 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: finishing write tid 2 to smithi017107884-28 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1685: finishing write tid 2 to smithi017107884-24 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: finishing write tid 3 to smithi017107884-28 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1685: finishing write tid 3 to smithi017107884-24 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: finishing write tid 4 to smithi017107884-28 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: finishing write tid 5 to smithi017107884-28 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: oid 28 updating version 0 to 103 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: oid 28 version 103 is already newer than 103 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: oid 28 version 103 is already newer than 103 2023-03-29T14:09:19.233 INFO:tasks.rados.rados.0.smithi017.stdout:1683: oid 28 version 103 is already newer than 99 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 103 (ObjNum 810 snap 196 seq_num 810) dirty exists 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1683: left oid 28 (ObjNum 810 snap 196 seq_num 810) 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: finishing write tid 1 to smithi017107884-15 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1685: finishing write tid 4 to smithi017107884-24 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: finishing write tid 2 to smithi017107884-15 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1685: finishing write tid 5 to smithi017107884-24 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1685: oid 24 updating version 0 to 103 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1685: oid 24 version 103 is already newer than 99 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1685: oid 24 version 103 is already newer than 103 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1685: oid 24 version 103 is already newer than 103 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 103 (ObjNum 811 snap 196 seq_num 811) dirty exists 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1685: left oid 24 (ObjNum 811 snap 196 seq_num 811) 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: finishing write tid 3 to smithi017107884-15 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: finishing write tid 4 to smithi017107884-15 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: finishing write tid 5 to smithi017107884-15 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: oid 15 updating version 0 to 348 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: oid 15 version 348 is already newer than 348 2023-03-29T14:09:19.234 INFO:tasks.rados.rados.0.smithi017.stdout:1686: oid 15 version 348 is already newer than 347 2023-03-29T14:09:19.235 INFO:tasks.rados.rados.0.smithi017.stdout:1686: oid 15 version 348 is already newer than 348 2023-03-29T14:09:19.235 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 348 (ObjNum 812 snap 196 seq_num 812) dirty exists 2023-03-29T14:09:19.235 INFO:tasks.rados.rados.0.smithi017.stdout:1686: left oid 15 (ObjNum 812 snap 196 seq_num 812) 2023-03-29T14:09:19.367 INFO:tasks.rados.rados.0.smithi017.stdout:1682: done (5 left) 2023-03-29T14:09:19.367 INFO:tasks.rados.rados.0.smithi017.stdout:1683: done (4 left) 2023-03-29T14:09:19.367 INFO:tasks.rados.rados.0.smithi017.stdout:1684: done (3 left) 2023-03-29T14:09:19.367 INFO:tasks.rados.rados.0.smithi017.stdout:1685: done (2 left) 2023-03-29T14:09:19.367 INFO:tasks.rados.rados.0.smithi017.stdout:1686: done (1 left) 2023-03-29T14:09:19.367 INFO:tasks.rados.rados.0.smithi017.stdout:1687: done (0 left) 2023-03-29T14:09:19.367 INFO:tasks.rados.rados.0.smithi017.stdout:1688: read oid 46 snap -1 2023-03-29T14:09:19.368 INFO:tasks.rados.rados.0.smithi017.stdout:1688: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:19.368 INFO:tasks.rados.rados.0.smithi017.stdout:1689: delete oid 41 current snap is 197 2023-03-29T14:09:19.447 INFO:tasks.rados.rados.0.smithi017.stdout:1689: done (1 left) 2023-03-29T14:09:19.447 INFO:tasks.rados.rados.0.smithi017.stdout:1690: write oid 24 current snap is 197 2023-03-29T14:09:19.447 INFO:tasks.rados.rados.0.smithi017.stdout:1690: seq_num 814 ranges {480898=502457,1699872=737281,3087022=593876} 2023-03-29T14:09:19.453 INFO:tasks.rados.rados.0.smithi017.stdout:1690: writing smithi017107884-24 from 480898 to 983355 tid 1 2023-03-29T14:09:19.460 INFO:tasks.rados.rados.0.smithi017.stdout:1690: writing smithi017107884-24 from 1699872 to 2437153 tid 2 2023-03-29T14:09:19.468 INFO:tasks.rados.rados.0.smithi017.stdout:1690: writing smithi017107884-24 from 3087022 to 3680898 tid 3 2023-03-29T14:09:19.468 INFO:tasks.rados.rados.0.smithi017.stdout:1691: write (excl) oid 31 current snap is 197 2023-03-29T14:09:19.468 INFO:tasks.rados.rados.0.smithi017.stdout:1691: seq_num 815 ranges {497705=577706,1642001=480939,2598092=497514} 2023-03-29T14:09:19.474 INFO:tasks.rados.rados.0.smithi017.stdout:1691: writing smithi017107884-31 from 497705 to 1075411 tid 1 2023-03-29T14:09:19.480 INFO:tasks.rados.rados.0.smithi017.stdout:1691: writing smithi017107884-31 from 1642001 to 2122940 tid 2 2023-03-29T14:09:19.485 INFO:tasks.rados.rados.0.smithi017.stdout:1691: writing smithi017107884-31 from 2598092 to 3095606 tid 3 2023-03-29T14:09:19.486 INFO:tasks.rados.rados.0.smithi017.stdout:1692: write (excl) oid 16 current snap is 197 2023-03-29T14:09:19.486 INFO:tasks.rados.rados.0.smithi017.stdout:1692: seq_num 816 ranges {514512=652955,1700483=740950,3025515=688997} 2023-03-29T14:09:19.492 INFO:tasks.rados.rados.0.smithi017.stdout:1692: writing smithi017107884-16 from 514512 to 1167467 tid 1 2023-03-29T14:09:19.499 INFO:tasks.rados.rados.0.smithi017.stdout:1692: writing smithi017107884-16 from 1700483 to 2441433 tid 2 2023-03-29T14:09:19.506 INFO:tasks.rados.rados.0.smithi017.stdout:1692: writing smithi017107884-16 from 3025515 to 3714512 tid 3 2023-03-29T14:09:19.506 INFO:tasks.rados.rados.0.smithi017.stdout:1693: write (excl) oid 15 current snap is 197 2023-03-29T14:09:19.506 INFO:tasks.rados.rados.0.smithi017.stdout:1688: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:19.627 INFO:tasks.rados.rados.0.smithi017.stdout:1690: finishing write tid 1 to smithi017107884-24 2023-03-29T14:09:19.627 INFO:tasks.rados.rados.0.smithi017.stdout:1690: finishing write tid 2 to smithi017107884-24 2023-03-29T14:09:19.627 INFO:tasks.rados.rados.0.smithi017.stdout:1691: finishing write tid 1 to smithi017107884-31 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: finishing write tid 3 to smithi017107884-24 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: finishing write tid 4 to smithi017107884-24 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: finishing write tid 5 to smithi017107884-24 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: oid 24 updating version 0 to 107 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: oid 24 updating version 107 to 108 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: oid 24 version 108 is already newer than 108 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: oid 24 version 108 is already newer than 108 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 108 (ObjNum 814 snap 197 seq_num 814) dirty exists 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1690: left oid 24 (ObjNum 814 snap 197 seq_num 814) 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1692: finishing write tid 1 to smithi017107884-16 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1691: finishing write tid 2 to smithi017107884-31 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1691: finishing write tid 3 to smithi017107884-31 2023-03-29T14:09:19.628 INFO:tasks.rados.rados.0.smithi017.stdout:1691: finishing write tid 4 to smithi017107884-31 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:1691: finishing write tid 5 to smithi017107884-31 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:1691: oid 31 updating version 0 to 156 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:1691: oid 31 version 156 is already newer than 152 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:1691: oid 31 version 156 is already newer than 156 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:1691: oid 31 version 156 is already newer than 156 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 156 (ObjNum 815 snap 197 seq_num 815) dirty exists 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:1691: left oid 31 (ObjNum 815 snap 197 seq_num 815) 2023-03-29T14:09:19.629 INFO:tasks.rados.rados.0.smithi017.stdout:1693: seq_num 817 ranges {531319=728204,1758965=484608,2652938=520058} 2023-03-29T14:09:19.634 INFO:tasks.rados.rados.0.smithi017.stdout:1693: writing smithi017107884-15 from 531319 to 1259523 tid 1 2023-03-29T14:09:19.639 INFO:tasks.rados.rados.0.smithi017.stdout:1693: writing smithi017107884-15 from 1758965 to 2243573 tid 2 2023-03-29T14:09:19.645 INFO:tasks.rados.rados.0.smithi017.stdout:1693: writing smithi017107884-15 from 2652938 to 3172996 tid 3 2023-03-29T14:09:19.645 INFO:tasks.rados.rados.0.smithi017.stdout:1688: done (4 left) 2023-03-29T14:09:19.645 INFO:tasks.rados.rados.0.smithi017.stdout:1690: done (3 left) 2023-03-29T14:09:19.645 INFO:tasks.rados.rados.0.smithi017.stdout:1691: done (2 left) 2023-03-29T14:09:19.645 INFO:tasks.rados.rados.0.smithi017.stdout:1694: read oid 19 snap 194 2023-03-29T14:09:19.645 INFO:tasks.rados.rados.0.smithi017.stdout:1694: expect (ObjNum 777 snap 189 seq_num 777) 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1695: delete oid 5 current snap is 197 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: finishing write tid 2 to smithi017107884-16 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: finishing write tid 3 to smithi017107884-16 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: finishing write tid 4 to smithi017107884-16 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: finishing write tid 5 to smithi017107884-16 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: oid 16 updating version 0 to 129 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: oid 16 version 129 is already newer than 129 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: oid 16 version 129 is already newer than 129 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: oid 16 version 129 is already newer than 125 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 129 (ObjNum 816 snap 197 seq_num 816) dirty exists 2023-03-29T14:09:19.646 INFO:tasks.rados.rados.0.smithi017.stdout:1692: left oid 16 (ObjNum 816 snap 197 seq_num 816) 2023-03-29T14:09:19.696 INFO:tasks.rados.rados.0.smithi017.stdout:1693: finishing write tid 1 to smithi017107884-15 2023-03-29T14:09:19.703 INFO:tasks.rados.rados.0.smithi017.stdout:1692: done (3 left) 2023-03-29T14:09:19.703 INFO:tasks.rados.rados.0.smithi017.stdout:1695: done (2 left) 2023-03-29T14:09:19.703 INFO:tasks.rados.rados.0.smithi017.stdout:1696: snap_remove snap 193 2023-03-29T14:09:19.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:20.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:20.048+0000 7f58f8fdc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:20.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:20.053+0000 7f58f8fdc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:20.120 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:20.121 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:20.370 INFO:tasks.rados.rados.0.smithi017.stdout:1696: done (2 left) 2023-03-29T14:09:20.370 INFO:tasks.rados.rados.0.smithi017.stdout:1697: write oid 1 current snap is 197 2023-03-29T14:09:20.370 INFO:tasks.rados.rados.0.smithi017.stdout:1697: seq_num 819 ranges {564933=478702,1759576=488277,2991431=658955} 2023-03-29T14:09:20.377 INFO:tasks.rados.rados.0.smithi017.stdout:1697: writing smithi017107884-1 from 564933 to 1043635 tid 1 2023-03-29T14:09:20.382 INFO:tasks.rados.rados.0.smithi017.stdout:1697: writing smithi017107884-1 from 1759576 to 2247853 tid 2 2023-03-29T14:09:20.390 INFO:tasks.rados.rados.0.smithi017.stdout:1697: writing smithi017107884-1 from 2991431 to 3650386 tid 3 2023-03-29T14:09:20.390 INFO:tasks.rados.rados.0.smithi017.stdout:1698: read oid 40 snap -1 2023-03-29T14:09:20.390 INFO:tasks.rados.rados.0.smithi017.stdout:1698: expect (ObjNum 766 snap 188 seq_num 766) 2023-03-29T14:09:20.391 INFO:tasks.rados.rados.0.smithi017.stdout:1699: write oid 4 current snap is 197 2023-03-29T14:09:20.391 INFO:tasks.rados.rados.0.smithi017.stdout:1693: finishing write tid 2 to smithi017107884-15 2023-03-29T14:09:20.391 INFO:tasks.rados.rados.0.smithi017.stdout:1699: seq_num 820 ranges {581740=553951,1818058=748288,3018854=470227} 2023-03-29T14:09:20.396 INFO:tasks.rados.rados.0.smithi017.stdout:1699: writing smithi017107884-4 from 581740 to 1135691 tid 1 2023-03-29T14:09:20.404 INFO:tasks.rados.rados.0.smithi017.stdout:1699: writing smithi017107884-4 from 1818058 to 2566346 tid 2 2023-03-29T14:09:20.410 INFO:tasks.rados.rados.0.smithi017.stdout:1699: writing smithi017107884-4 from 3018854 to 3489081 tid 3 2023-03-29T14:09:20.410 INFO:tasks.rados.rados.0.smithi017.stdout:1693: finishing write tid 3 to smithi017107884-15 2023-03-29T14:09:20.410 INFO:tasks.rados.rados.0.smithi017.stdout:1694: expect (ObjNum 777 snap 189 seq_num 777) 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: finishing write tid 4 to smithi017107884-15 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: finishing write tid 5 to smithi017107884-15 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: oid 15 updating version 0 to 353 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: oid 15 version 353 is already newer than 353 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: oid 15 version 353 is already newer than 349 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: oid 15 version 353 is already newer than 353 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 353 (ObjNum 817 snap 197 seq_num 817) dirty exists 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: left oid 15 (ObjNum 817 snap 197 seq_num 817) 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1700: read oid 21 snap -1 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1700: expect (ObjNum 712 snap 175 seq_num 712) 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1693: done (5 left) 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1694: done (4 left) 2023-03-29T14:09:20.581 INFO:tasks.rados.rados.0.smithi017.stdout:1701: read oid 28 snap -1 2023-03-29T14:09:20.582 INFO:tasks.rados.rados.0.smithi017.stdout:1701: expect (ObjNum 810 snap 196 seq_num 810) 2023-03-29T14:09:20.582 INFO:tasks.rados.rados.0.smithi017.stdout:1702: snap_create 2023-03-29T14:09:20.582 INFO:tasks.rados.rados.0.smithi017.stdout:1698: expect (ObjNum 766 snap 188 seq_num 766) 2023-03-29T14:09:20.788 INFO:tasks.rados.rados.0.smithi017.stdout:1699: finishing write tid 1 to smithi017107884-4 2023-03-29T14:09:20.789 INFO:tasks.rados.rados.0.smithi017.stdout:1697: finishing write tid 1 to smithi017107884-1 2023-03-29T14:09:20.789 INFO:tasks.rados.rados.0.smithi017.stdout:1697: finishing write tid 2 to smithi017107884-1 2023-03-29T14:09:20.789 INFO:tasks.rados.rados.0.smithi017.stdout:1697: finishing write tid 3 to smithi017107884-1 2023-03-29T14:09:20.789 INFO:tasks.rados.rados.0.smithi017.stdout:1697: finishing write tid 4 to smithi017107884-1 2023-03-29T14:09:20.789 INFO:tasks.rados.rados.0.smithi017.stdout:1697: finishing write tid 5 to smithi017107884-1 2023-03-29T14:09:20.790 INFO:tasks.rados.rados.0.smithi017.stdout:1697: oid 1 updating version 0 to 232 2023-03-29T14:09:20.790 INFO:tasks.rados.rados.0.smithi017.stdout:1697: oid 1 version 232 is already newer than 232 2023-03-29T14:09:20.790 INFO:tasks.rados.rados.0.smithi017.stdout:1697: oid 1 version 232 is already newer than 232 2023-03-29T14:09:20.790 INFO:tasks.rados.rados.0.smithi017.stdout:1697: oid 1 version 232 is already newer than 232 2023-03-29T14:09:20.790 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 232 (ObjNum 819 snap 197 seq_num 819) dirty exists 2023-03-29T14:09:20.790 INFO:tasks.rados.rados.0.smithi017.stdout:1697: left oid 1 (ObjNum 819 snap 197 seq_num 819) 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: finishing write tid 2 to smithi017107884-4 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: finishing write tid 3 to smithi017107884-4 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: finishing write tid 4 to smithi017107884-4 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: finishing write tid 5 to smithi017107884-4 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: oid 4 updating version 0 to 260 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: oid 4 version 260 is already newer than 260 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: oid 4 version 260 is already newer than 260 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:1699: oid 4 version 260 is already newer than 260 2023-03-29T14:09:20.791 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 4 v 260 (ObjNum 820 snap 197 seq_num 820) dirty exists 2023-03-29T14:09:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:1699: left oid 4 (ObjNum 820 snap 197 seq_num 820) 2023-03-29T14:09:20.792 INFO:tasks.rados.rados.0.smithi017.stdout:1700: expect (ObjNum 712 snap 175 seq_num 712) 2023-03-29T14:09:20.941 INFO:tasks.rados.rados.0.smithi017.stdout:1701: expect (ObjNum 810 snap 196 seq_num 810) 2023-03-29T14:09:21.374 INFO:tasks.rados.rados.0.smithi017.stdout:1697: done (5 left) 2023-03-29T14:09:21.374 INFO:tasks.rados.rados.0.smithi017.stdout:1698: done (4 left) 2023-03-29T14:09:21.374 INFO:tasks.rados.rados.0.smithi017.stdout:1699: done (3 left) 2023-03-29T14:09:21.374 INFO:tasks.rados.rados.0.smithi017.stdout:1700: done (2 left) 2023-03-29T14:09:21.374 INFO:tasks.rados.rados.0.smithi017.stdout:1701: done (1 left) 2023-03-29T14:09:21.375 INFO:tasks.rados.rados.0.smithi017.stdout:1702: done (0 left) 2023-03-29T14:09:21.375 INFO:tasks.rados.rados.0.smithi017.stdout:1703: read oid 25 snap -1 2023-03-29T14:09:21.375 INFO:tasks.rados.rados.0.smithi017.stdout:1703: expect (ObjNum 793 snap 193 seq_num 793) 2023-03-29T14:09:21.375 INFO:tasks.rados.rados.0.smithi017.stdout:1704: read oid 32 snap 194 2023-03-29T14:09:21.375 INFO:tasks.rados.rados.0.smithi017.stdout:1704: expect (ObjNum 667 snap 163 seq_num 667) 2023-03-29T14:09:21.375 INFO:tasks.rados.rados.0.smithi017.stdout:1705: snap_remove snap 169 2023-03-29T14:09:22.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:22.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:22.248+0000 7f0a9d5bf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:22.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:22.253+0000 7f0a9d5bf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:22.316 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:22.319 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:22.320 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:22.378 INFO:tasks.rados.rados.0.smithi017.stdout:1705: done (2 left) 2023-03-29T14:09:22.378 INFO:tasks.rados.rados.0.smithi017.stdout:1706: read oid 47 snap 196 2023-03-29T14:09:22.379 INFO:tasks.rados.rados.0.smithi017.stdout:1706: expect deleted 2023-03-29T14:09:22.379 INFO:tasks.rados.rados.0.smithi017.stdout:1707: delete oid 13 current snap is 198 2023-03-29T14:09:22.379 INFO:tasks.rados.rados.0.smithi017.stdout:1703: expect (ObjNum 793 snap 193 seq_num 793) 2023-03-29T14:09:22.445 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:22.446 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:22.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:22.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:22.574+0000 7f90bd38d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:22.582 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:22.581+0000 7f90bd38d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:22.597 INFO:tasks.rados.rados.0.smithi017.stdout:1704: expect (ObjNum 667 snap 163 seq_num 667) 2023-03-29T14:09:22.787 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:22.801 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":627,"stamp":"2023-03-29T14:09:22.373483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266643562,"num_objects":956,"num_object_clones":860,"num_object_copies":2868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":3860,"num_read_kb":2283920,"num_write":5300,"num_write_kb":1469092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2796,"ondisk_log_size":2796,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1852108,"kb_used_data":1768044,"kb_used_omap":0,"kb_used_meta":84032,"kb_avail":375635252,"statfs":{"total":386547056640,"available":384650498048,"internally_reserved":0,"allocated":1810477056,"data_stored":2244163878,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":86048768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30219712,"num_objects":28,"num_object_clones":28,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-4,"num_read":141,"num_read_kb":102149,"num_write":160,"num_write_kb":53070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012248"},"pg_stats":[{"pgid":"3.f","version":"412'129","reported_seq":409,"reported_epoch":414,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:20.389156+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:20.389156+0000","last_peered":"2023-03-29T14:09:20.389156+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:20.389156+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126894,"num_objects":28,"num_object_clones":28,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":65946,"num_write":172,"num_write_kb":51352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":442,"reported_epoch":412,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:18.384617+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:18.384617+0000","last_peered":"2023-03-29T14:09:18.384617+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:18.384617+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"413'260","reported_seq":936,"reported_epoch":414,"state":"active+clean","last_fresh":"2023-03-29T14:09:20.567727+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:20.567727+0000","last_peered":"2023-03-29T14:09:20.567727+0000","last_clean":"2023-03-29T14:09:20.567727+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:20.567727+0000","last_undegraded":"2023-03-29T14:09:20.567727+0000","last_fullsized":"2023-03-29T14:09:20.567727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808848,"num_objects":94,"num_object_clones":84,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":275565,"num_write":504,"num_write_kb":150092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"406'273","reported_seq":949,"reported_epoch":414,"state":"active+clean","last_fresh":"2023-03-29T14:09:21.408797+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:21.408797+0000","last_peered":"2023-03-29T14:09:21.408797+0000","last_clean":"2023-03-29T14:09:21.408797+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:21.408797+0000","last_undegraded":"2023-03-29T14:09:21.408797+0000","last_fullsized":"2023-03-29T14:09:21.408797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346276,"num_objects":108,"num_object_clones":100,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":377,"num_read_kb":226490,"num_write":536,"num_write_kb":151626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"410'222","reported_seq":821,"reported_epoch":412,"state":"active+clean","last_fresh":"2023-03-29T14:09:19.464329+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:19.464329+0000","last_peered":"2023-03-29T14:09:19.464329+0000","last_clean":"2023-03-29T14:09:19.464329+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:19.464329+0000","last_undegraded":"2023-03-29T14:09:19.464329+0000","last_fullsized":"2023-03-29T14:09:19.464329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23127686,"num_objects":78,"num_object_clones":70,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":320,"num_read_kb":206202,"num_write":444,"num_write_kb":130902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":350,"reported_epoch":414,"state":"active+undersized","last_fresh":"2023-03-29T14:09:20.387149+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:20.387149+0000","last_peered":"2023-03-29T14:09:20.387149+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:20.387149+0000","last_undegraded":"2023-03-29T14:09:20.387149+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"412'156","reported_seq":672,"reported_epoch":414,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:20.390637+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:20.390637+0000","last_peered":"2023-03-29T14:09:20.390637+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:20.390637+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19329040,"num_objects":54,"num_object_clones":48,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":74216,"num_write":286,"num_write_kb":80828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":54}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":570,"reported_epoch":414,"state":"active+clean","last_fresh":"2023-03-29T14:09:20.616251+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:20.616251+0000","last_peered":"2023-03-29T14:09:20.616251+0000","last_clean":"2023-03-29T14:09:20.616251+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:20.616251+0000","last_undegraded":"2023-03-29T14:09:20.616251+0000","last_fullsized":"2023-03-29T14:09:20.616251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":73865,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":412,"state":"active+undersized","last_fresh":"2023-03-29T14:09:18.418644+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:18.418644+0000","last_peered":"2023-03-29T14:09:18.418644+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:18.418644+0000","last_undegraded":"2023-03-29T14:09:18.418644+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"390'98","reported_seq":315,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:19.390174+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:19.390174+0000","last_peered":"2023-03-29T14:09:19.390174+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:19.390174+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084860,"num_objects":20,"num_object_clones":20,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":26577,"num_write":120,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":414,"state":"active+undersized","last_fresh":"2023-03-29T14:09:20.392187+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:20.392187+0000","last_peered":"2023-03-29T14:09:20.392187+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:20.392187+0000","last_undegraded":"2023-03-29T14:09:20.392187+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"412'108","reported_seq":573,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:19.563848+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:19.563848+0000","last_peered":"2023-03-29T14:09:19.563848+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:19.563848+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7361796,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":165,"num_read_kb":107605,"num_write":210,"num_write_kb":62886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":412,"state":"active+undersized","last_fresh":"2023-03-29T14:09:18.428583+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:18.428583+0000","last_peered":"2023-03-29T14:09:18.428583+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:18.428583+0000","last_undegraded":"2023-03-29T14:09:18.428583+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"410'155","reported_seq":677,"reported_epoch":412,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:18.480571+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:18.480571+0000","last_peered":"2023-03-29T14:09:18.480571+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:18.480571+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18791150,"num_objects":56,"num_object_clones":50,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":149325,"num_write":298,"num_write_kb":88962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":353,"reported_epoch":414,"state":"active+undersized","last_fresh":"2023-03-29T14:09:20.386432+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:20.386432+0000","last_peered":"2023-03-29T14:09:20.386432+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:20.386432+0000","last_undegraded":"2023-03-29T14:09:20.386432+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":546,"reported_epoch":412,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:18.397598+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:18.397598+0000","last_peered":"2023-03-29T14:09:18.397598+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:18.397598+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"393'198","reported_seq":839,"reported_epoch":414,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:20.606105+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:20.606105+0000","last_peered":"2023-03-29T14:09:20.606105+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:20.606105+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19900412,"num_objects":72,"num_object_clones":64,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":368,"num_read_kb":253750,"num_write":384,"num_write_kb":111100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":72}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":353,"reported_epoch":414,"state":"active+undersized","last_fresh":"2023-03-29T14:09:20.387885+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:20.387885+0000","last_peered":"2023-03-29T14:09:20.387885+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:20.387885+0000","last_undegraded":"2023-03-29T14:09:20.387885+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"412'353","reported_seq":1126,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:19.758249+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:19.758249+0000","last_peered":"2023-03-29T14:09:19.758249+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:19.758249+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33850816,"num_objects":126,"num_object_clones":112,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":534,"num_read_kb":327697,"num_write":662,"num_write_kb":193330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":126}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":412,"state":"active+undersized","last_fresh":"2023-03-29T14:09:18.414117+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:18.414117+0000","last_peered":"2023-03-29T14:09:18.414117+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:18.414117+0000","last_undegraded":"2023-03-29T14:09:18.414117+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":576,"reported_epoch":412,"state":"active+clean","last_fresh":"2023-03-29T14:09:18.420486+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:18.420486+0000","last_peered":"2023-03-29T14:09:18.420486+0000","last_clean":"2023-03-29T14:09:18.420486+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:18.420486+0000","last_undegraded":"2023-03-29T14:09:18.420486+0000","last_fullsized":"2023-03-29T14:09:18.420486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":145,"num_read_kb":79798,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"411'231","reported_seq":812,"reported_epoch":412,"state":"active+clean","last_fresh":"2023-03-29T14:09:19.651173+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:19.651173+0000","last_peered":"2023-03-29T14:09:19.651173+0000","last_clean":"2023-03-29T14:09:19.651173+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:19.651173+0000","last_undegraded":"2023-03-29T14:09:19.651173+0000","last_fullsized":"2023-03-29T14:09:19.651173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11765560,"num_objects":84,"num_object_clones":74,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":292,"num_read_kb":172365,"num_write":422,"num_write_kb":122536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":321,"reported_epoch":412,"state":"active+undersized","last_fresh":"2023-03-29T14:09:18.425189+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:18.425189+0000","last_peered":"2023-03-29T14:09:18.425189+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:18.425189+0000","last_undegraded":"2023-03-29T14:09:18.425189+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":353,"reported_epoch":414,"state":"active+undersized","last_fresh":"2023-03-29T14:09:20.377034+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:20.377034+0000","last_peered":"2023-03-29T14:09:20.377034+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:20.377034+0000","last_undegraded":"2023-03-29T14:09:20.377034+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"413'232","reported_seq":847,"reported_epoch":413,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:20.518335+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:20.518335+0000","last_peered":"2023-03-29T14:09:20.518335+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:20.518335+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20270232,"num_objects":84,"num_object_clones":76,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":337,"num_read_kb":213657,"num_write":440,"num_write_kb":130642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":84}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":265462826,"num_objects":952,"num_object_clones":860,"num_object_copies":2856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":18,"num_read":3718,"num_read_kb":2283800,"num_write":5026,"num_write_kb":1465304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2699,"ondisk_log_size":2699,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":541964,"kb_used_data":518596,"kb_used_omap":0,"kb_used_meta":23360,"kb_avail":93829876,"statfs":{"total":96636764160,"available":96081793024,"internally_reserved":0,"allocated":531042304,"data_stored":661312956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23920640},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706098,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":605388,"kb_used_data":580612,"kb_used_omap":0,"kb_used_meta":24768,"kb_avail":93766452,"statfs":{"total":96636764160,"available":96016846848,"internally_reserved":0,"allocated":594546688,"data_stored":740130776,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25362432},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706098,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":548812,"kb_used_data":524996,"kb_used_omap":0,"kb_used_meta":23808,"kb_avail":93823028,"statfs":{"total":96636764160,"available":96074780672,"internally_reserved":0,"allocated":537595904,"data_stored":669487921,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24379392},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:22.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.e 3.9 1.0 3.1 2023-03-29T14:09:22.806 INFO:tasks.rados.rados.0.smithi017.stdout:1703: done (3 left) 2023-03-29T14:09:22.806 INFO:tasks.rados.rados.0.smithi017.stdout:1704: done (2 left) 2023-03-29T14:09:22.807 INFO:tasks.rados.rados.0.smithi017.stdout:1707: done (1 left) 2023-03-29T14:09:22.807 INFO:tasks.rados.rados.0.smithi017.stdout:1708: write oid 11 current snap is 198 2023-03-29T14:09:22.807 INFO:tasks.rados.rados.0.smithi017.stdout:1708: seq_num 822 ranges {615354=420802,1651375=751957,3190053=492771} 2023-03-29T14:09:22.807 INFO:tasks.rados.rados.0.smithi017.stdout:1708: writing smithi017107884-11 from 615354 to 1036156 tid 1 2023-03-29T14:09:22.815 INFO:tasks.rados.rados.0.smithi017.stdout:1708: writing smithi017107884-11 from 1651375 to 2403332 tid 2 2023-03-29T14:09:22.822 INFO:tasks.rados.rados.0.smithi017.stdout:1708: writing smithi017107884-11 from 3190053 to 3682824 tid 3 2023-03-29T14:09:22.822 INFO:tasks.rados.rados.0.smithi017.stdout:1706: done (1 left) 2023-03-29T14:09:22.823 INFO:tasks.rados.rados.0.smithi017.stdout:1709: read oid 9 snap 195 2023-03-29T14:09:22.823 INFO:tasks.rados.rados.0.smithi017.stdout:1709: expect (ObjNum 788 snap 191 seq_num 788) 2023-03-29T14:09:22.823 INFO:tasks.rados.rados.0.smithi017.stdout:1710: write oid 29 current snap is 198 2023-03-29T14:09:22.823 INFO:tasks.rados.rados.0.smithi017.stdout:1710: seq_num 823 ranges {632161=496051,1593504=495615,2701123=420396,3611645=220516} 2023-03-29T14:09:22.829 INFO:tasks.rados.rados.0.smithi017.stdout:1710: writing smithi017107884-29 from 632161 to 1128212 tid 1 2023-03-29T14:09:22.834 INFO:tasks.rados.rados.0.smithi017.stdout:1710: writing smithi017107884-29 from 1593504 to 2089119 tid 2 2023-03-29T14:09:22.840 INFO:tasks.rados.rados.0.smithi017.stdout:1710: writing smithi017107884-29 from 2701123 to 3121519 tid 3 2023-03-29T14:09:22.845 INFO:tasks.rados.rados.0.smithi017.stdout:1710: writing smithi017107884-29 from 3611645 to 3832161 tid 4 2023-03-29T14:09:22.845 INFO:tasks.rados.rados.0.smithi017.stdout:1711: write oid 19 current snap is 198 2023-03-29T14:09:22.845 INFO:tasks.rados.rados.0.smithi017.stdout:1711: seq_num 824 ranges {648968=571300,1651986=639273,2728546=631668} 2023-03-29T14:09:22.851 INFO:tasks.rados.rados.0.smithi017.stdout:1711: writing smithi017107884-19 from 648968 to 1220268 tid 1 2023-03-29T14:09:22.857 INFO:tasks.rados.rados.0.smithi017.stdout:1711: writing smithi017107884-19 from 1651986 to 2291259 tid 2 2023-03-29T14:09:22.863 INFO:tasks.rados.rados.0.smithi017.stdout:1711: writing smithi017107884-19 from 2728546 to 3360214 tid 3 2023-03-29T14:09:22.863 INFO:tasks.rados.rados.0.smithi017.stdout:1712: snap_remove snap 194 2023-03-29T14:09:22.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:22.942+0000 7feef3819700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:22.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:22.955+0000 7feef3819700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:23.166 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1] on osd.0 to force-recovery; instructing pg(s) [1.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.9] on osd.3 to force-recovery; 2023-03-29T14:09:23.166 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:09:23.253 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~253s 2023-03-29T14:09:23.260 INFO:tasks.rados.rados.0.smithi017.stdout:1712: done (4 left) 2023-03-29T14:09:23.260 INFO:tasks.rados.rados.0.smithi017.stdout:1713: read oid 43 snap -1 2023-03-29T14:09:23.260 INFO:tasks.rados.rados.0.smithi017.stdout:1713: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:23.260 INFO:tasks.rados.rados.0.smithi017.stdout:1714: snap_create 2023-03-29T14:09:23.260 INFO:tasks.rados.rados.0.smithi017.stdout:1709: expect (ObjNum 788 snap 191 seq_num 788) 2023-03-29T14:09:23.562 INFO:tasks.rados.rados.0.smithi017.stdout:1708: finishing write tid 1 to smithi017107884-11 2023-03-29T14:09:23.562 INFO:tasks.rados.rados.0.smithi017.stdout:1708: finishing write tid 2 to smithi017107884-11 2023-03-29T14:09:23.562 INFO:tasks.rados.rados.0.smithi017.stdout:1708: finishing write tid 3 to smithi017107884-11 2023-03-29T14:09:23.562 INFO:tasks.rados.rados.0.smithi017.stdout:1708: finishing write tid 4 to smithi017107884-11 2023-03-29T14:09:23.562 INFO:tasks.rados.rados.0.smithi017.stdout:1708: finishing write tid 5 to smithi017107884-11 2023-03-29T14:09:23.562 INFO:tasks.rados.rados.0.smithi017.stdout:1708: oid 11 updating version 0 to 358 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1708: oid 11 version 358 is already newer than 356 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1708: oid 11 version 358 is already newer than 358 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1708: oid 11 version 358 is already newer than 358 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 11 v 358 (ObjNum 822 snap 198 seq_num 822) dirty exists 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1708: left oid 11 (ObjNum 822 snap 198 seq_num 822) 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1710: finishing write tid 1 to smithi017107884-29 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1710: finishing write tid 2 to smithi017107884-29 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1710: finishing write tid 3 to smithi017107884-29 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1710: finishing write tid 4 to smithi017107884-29 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1710: finishing write tid 5 to smithi017107884-29 2023-03-29T14:09:23.563 INFO:tasks.rados.rados.0.smithi017.stdout:1710: finishing write tid 6 to smithi017107884-29 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1710: oid 29 updating version 0 to 113 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1710: oid 29 version 113 is already newer than 111 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1710: oid 29 version 113 is already newer than 113 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1710: oid 29 version 113 is already newer than 112 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1710: oid 29 version 113 is already newer than 110 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 113 (ObjNum 823 snap 198 seq_num 823) dirty exists 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1710: left oid 29 (ObjNum 823 snap 198 seq_num 823) 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1711: finishing write tid 1 to smithi017107884-19 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1711: finishing write tid 2 to smithi017107884-19 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1711: finishing write tid 3 to smithi017107884-19 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1711: finishing write tid 4 to smithi017107884-19 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1711: finishing write tid 5 to smithi017107884-19 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1711: oid 19 updating version 0 to 237 2023-03-29T14:09:23.564 INFO:tasks.rados.rados.0.smithi017.stdout:1711: oid 19 version 237 is already newer than 236 2023-03-29T14:09:23.565 INFO:tasks.rados.rados.0.smithi017.stdout:1711: oid 19 version 237 is already newer than 237 2023-03-29T14:09:23.565 INFO:tasks.rados.rados.0.smithi017.stdout:1711: oid 19 version 237 is already newer than 237 2023-03-29T14:09:23.565 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 237 (ObjNum 824 snap 198 seq_num 824) dirty exists 2023-03-29T14:09:23.565 INFO:tasks.rados.rados.0.smithi017.stdout:1711: left oid 19 (ObjNum 824 snap 198 seq_num 824) 2023-03-29T14:09:23.565 INFO:tasks.rados.rados.0.smithi017.stdout:1713: expect (ObjNum 619 snap 145 seq_num 619) 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1708: done (5 left) 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1709: done (4 left) 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1710: done (3 left) 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1711: done (2 left) 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1713: done (1 left) 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1714: done (0 left) 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1715: write oid 26 current snap is 199 2023-03-29T14:09:24.264 INFO:tasks.rados.rados.0.smithi017.stdout:1715: seq_num 825 ranges {665775=646549,2110468=499284,3155969=442940} 2023-03-29T14:09:24.271 INFO:tasks.rados.rados.0.smithi017.stdout:1715: writing smithi017107884-26 from 665775 to 1312324 tid 1 2023-03-29T14:09:24.278 INFO:tasks.rados.rados.0.smithi017.stdout:1715: writing smithi017107884-26 from 2110468 to 2609752 tid 2 2023-03-29T14:09:24.283 INFO:tasks.rados.rados.0.smithi017.stdout:1715: writing smithi017107884-26 from 3155969 to 3598909 tid 3 2023-03-29T14:09:24.283 INFO:tasks.rados.rados.0.smithi017.stdout:1716: delete oid 47 current snap is 199 2023-03-29T14:09:24.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:24.355 INFO:tasks.rados.rados.0.smithi017.stdout:1716: done (1 left) 2023-03-29T14:09:24.355 INFO:tasks.rados.rados.0.smithi017.stdout:1717: snap_create 2023-03-29T14:09:24.386 INFO:tasks.rados.rados.0.smithi017.stdout:1715: finishing write tid 1 to smithi017107884-26 2023-03-29T14:09:24.427 INFO:tasks.rados.rados.0.smithi017.stdout:1715: finishing write tid 2 to smithi017107884-26 2023-03-29T14:09:24.427 INFO:tasks.rados.rados.0.smithi017.stdout:1715: finishing write tid 3 to smithi017107884-26 2023-03-29T14:09:24.427 INFO:tasks.rados.rados.0.smithi017.stdout:1715: finishing write tid 4 to smithi017107884-26 2023-03-29T14:09:24.427 INFO:tasks.rados.rados.0.smithi017.stdout:1715: finishing write tid 5 to smithi017107884-26 2023-03-29T14:09:24.428 INFO:tasks.rados.rados.0.smithi017.stdout:1715: oid 26 updating version 0 to 203 2023-03-29T14:09:24.428 INFO:tasks.rados.rados.0.smithi017.stdout:1715: oid 26 version 203 is already newer than 203 2023-03-29T14:09:24.428 INFO:tasks.rados.rados.0.smithi017.stdout:1715: oid 26 version 203 is already newer than 203 2023-03-29T14:09:24.428 INFO:tasks.rados.rados.0.smithi017.stdout:1715: oid 26 version 203 is already newer than 203 2023-03-29T14:09:24.428 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 203 (ObjNum 825 snap 199 seq_num 825) dirty exists 2023-03-29T14:09:24.428 INFO:tasks.rados.rados.0.smithi017.stdout:1715: left oid 26 (ObjNum 825 snap 199 seq_num 825) 2023-03-29T14:09:24.453 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:24.451+0000 7f3f26374700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:24.455+0000 7f3f26374700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:24.520 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:24.521 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:25.264 INFO:tasks.rados.rados.0.smithi017.stdout:1715: done (1 left) 2023-03-29T14:09:25.264 INFO:tasks.rados.rados.0.smithi017.stdout:1717: done (0 left) 2023-03-29T14:09:25.264 INFO:tasks.rados.rados.0.smithi017.stdout:1718: write oid 21 current snap is 200 2023-03-29T14:09:25.264 INFO:tasks.rados.rados.0.smithi017.stdout:1718: seq_num 827 ranges {699389=797047,2111079=502953,3094462=581837} 2023-03-29T14:09:25.272 INFO:tasks.rados.rados.0.smithi017.stdout:1718: writing smithi017107884-21 from 699389 to 1496436 tid 1 2023-03-29T14:09:25.278 INFO:tasks.rados.rados.0.smithi017.stdout:1718: writing smithi017107884-21 from 2111079 to 2614032 tid 2 2023-03-29T14:09:25.284 INFO:tasks.rados.rados.0.smithi017.stdout:1718: writing smithi017107884-21 from 3094462 to 3676299 tid 3 2023-03-29T14:09:25.285 INFO:tasks.rados.rados.0.smithi017.stdout:1719: write (excl) oid 3 current snap is 200 2023-03-29T14:09:25.285 INFO:tasks.rados.rados.0.smithi017.stdout:1719: seq_num 828 ranges {716196=472296,1769561=646611,3121885=793109} 2023-03-29T14:09:25.291 INFO:tasks.rados.rados.0.smithi017.stdout:1719: writing smithi017107884-3 from 716196 to 1188492 tid 1 2023-03-29T14:09:25.298 INFO:tasks.rados.rados.0.smithi017.stdout:1719: writing smithi017107884-3 from 1769561 to 2416172 tid 2 2023-03-29T14:09:25.306 INFO:tasks.rados.rados.0.smithi017.stdout:1719: writing smithi017107884-3 from 3121885 to 3914994 tid 3 2023-03-29T14:09:25.306 INFO:tasks.rados.rados.0.smithi017.stdout:1720: write (excl) oid 37 current snap is 200 2023-03-29T14:09:25.306 INFO:tasks.rados.rados.0.smithi017.stdout:1720: seq_num 829 ranges {733003=663898,1944396=506622,2865661=604381} 2023-03-29T14:09:25.313 INFO:tasks.rados.rados.0.smithi017.stdout:1720: writing smithi017107884-37 from 733003 to 1396901 tid 1 2023-03-29T14:09:25.319 INFO:tasks.rados.rados.0.smithi017.stdout:1720: writing smithi017107884-37 from 1944396 to 2451018 tid 2 2023-03-29T14:09:25.325 INFO:tasks.rados.rados.0.smithi017.stdout:1720: writing smithi017107884-37 from 2865661 to 3470042 tid 3 2023-03-29T14:09:25.325 INFO:tasks.rados.rados.0.smithi017.stdout:1721: snap_create 2023-03-29T14:09:25.340 INFO:tasks.rados.rados.0.smithi017.stdout:1718: finishing write tid 1 to smithi017107884-21 2023-03-29T14:09:25.385 INFO:tasks.rados.rados.0.smithi017.stdout:1718: finishing write tid 2 to smithi017107884-21 2023-03-29T14:09:25.385 INFO:tasks.rados.rados.0.smithi017.stdout:1718: finishing write tid 3 to smithi017107884-21 2023-03-29T14:09:25.386 INFO:tasks.rados.rados.0.smithi017.stdout:1718: finishing write tid 4 to smithi017107884-21 2023-03-29T14:09:25.387 INFO:tasks.rados.rados.0.smithi017.stdout:1718: finishing write tid 5 to smithi017107884-21 2023-03-29T14:09:25.387 INFO:tasks.rados.rados.0.smithi017.stdout:1718: oid 21 updating version 0 to 208 2023-03-29T14:09:25.387 INFO:tasks.rados.rados.0.smithi017.stdout:1718: oid 21 version 208 is already newer than 208 2023-03-29T14:09:25.388 INFO:tasks.rados.rados.0.smithi017.stdout:1718: oid 21 version 208 is already newer than 208 2023-03-29T14:09:25.388 INFO:tasks.rados.rados.0.smithi017.stdout:1718: oid 21 version 208 is already newer than 208 2023-03-29T14:09:25.388 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 208 (ObjNum 827 snap 200 seq_num 827) dirty exists 2023-03-29T14:09:25.388 INFO:tasks.rados.rados.0.smithi017.stdout:1718: left oid 21 (ObjNum 827 snap 200 seq_num 827) 2023-03-29T14:09:25.403 INFO:tasks.rados.rados.0.smithi017.stdout:1719: finishing write tid 1 to smithi017107884-3 2023-03-29T14:09:25.442 INFO:tasks.rados.rados.0.smithi017.stdout:1720: finishing write tid 1 to smithi017107884-37 2023-03-29T14:09:25.461 INFO:tasks.rados.rados.0.smithi017.stdout:1719: finishing write tid 2 to smithi017107884-3 2023-03-29T14:09:25.472 INFO:tasks.rados.rados.0.smithi017.stdout:1719: finishing write tid 3 to smithi017107884-3 2023-03-29T14:09:25.473 INFO:tasks.rados.rados.0.smithi017.stdout:1719: finishing write tid 4 to smithi017107884-3 2023-03-29T14:09:25.477 INFO:tasks.rados.rados.0.smithi017.stdout:1719: finishing write tid 5 to smithi017107884-3 2023-03-29T14:09:25.477 INFO:tasks.rados.rados.0.smithi017.stdout:1719: oid 3 updating version 0 to 362 2023-03-29T14:09:25.478 INFO:tasks.rados.rados.0.smithi017.stdout:1719: oid 3 version 362 is already newer than 362 2023-03-29T14:09:25.478 INFO:tasks.rados.rados.0.smithi017.stdout:1719: oid 3 version 362 is already newer than 362 2023-03-29T14:09:25.478 INFO:tasks.rados.rados.0.smithi017.stdout:1719: oid 3 version 362 is already newer than 359 2023-03-29T14:09:25.478 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 362 (ObjNum 828 snap 200 seq_num 828) dirty exists 2023-03-29T14:09:25.478 INFO:tasks.rados.rados.0.smithi017.stdout:1719: left oid 3 (ObjNum 828 snap 200 seq_num 828) 2023-03-29T14:09:25.478 INFO:tasks.rados.rados.0.smithi017.stdout:1720: finishing write tid 2 to smithi017107884-37 2023-03-29T14:09:25.479 INFO:tasks.rados.rados.0.smithi017.stdout:1720: finishing write tid 3 to smithi017107884-37 2023-03-29T14:09:25.480 INFO:tasks.rados.rados.0.smithi017.stdout:1720: finishing write tid 4 to smithi017107884-37 2023-03-29T14:09:25.482 INFO:tasks.rados.rados.0.smithi017.stdout:1720: finishing write tid 5 to smithi017107884-37 2023-03-29T14:09:25.482 INFO:tasks.rados.rados.0.smithi017.stdout:1720: oid 37 updating version 0 to 103 2023-03-29T14:09:25.483 INFO:tasks.rados.rados.0.smithi017.stdout:1720: oid 37 version 103 is already newer than 99 2023-03-29T14:09:25.483 INFO:tasks.rados.rados.0.smithi017.stdout:1720: oid 37 version 103 is already newer than 103 2023-03-29T14:09:25.483 INFO:tasks.rados.rados.0.smithi017.stdout:1720: oid 37 version 103 is already newer than 103 2023-03-29T14:09:25.483 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 103 (ObjNum 829 snap 200 seq_num 829) dirty exists 2023-03-29T14:09:25.483 INFO:tasks.rados.rados.0.smithi017.stdout:1720: left oid 37 (ObjNum 829 snap 200 seq_num 829) 2023-03-29T14:09:26.265 INFO:tasks.rados.rados.0.smithi017.stdout:1718: done (3 left) 2023-03-29T14:09:26.265 INFO:tasks.rados.rados.0.smithi017.stdout:1719: done (2 left) 2023-03-29T14:09:26.265 INFO:tasks.rados.rados.0.smithi017.stdout:1720: done (1 left) 2023-03-29T14:09:26.265 INFO:tasks.rados.rados.0.smithi017.stdout:1721: done (0 left) 2023-03-29T14:09:26.265 INFO:tasks.rados.rados.0.smithi017.stdout:1722: snap_create 2023-03-29T14:09:26.522 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:26.648+0000 7f0863c80700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:26.652+0000 7f0863c80700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:26.716 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:26.719 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:26.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:27.266 INFO:tasks.rados.rados.0.smithi017.stdout:1722: done (0 left) 2023-03-29T14:09:27.266 INFO:tasks.rados.rados.0.smithi017.stdout:1723: write (excl) oid 19 current snap is 202 2023-03-29T14:09:27.266 INFO:tasks.rados.rados.0.smithi017.stdout:1723: seq_num 830 ranges {749810=739147,2002878=650280,3293084=415653} 2023-03-29T14:09:27.274 INFO:tasks.rados.rados.0.smithi017.stdout:1723: writing smithi017107884-19 from 749810 to 1488957 tid 1 2023-03-29T14:09:27.280 INFO:tasks.rados.rados.0.smithi017.stdout:1723: writing smithi017107884-19 from 2002878 to 2653158 tid 2 2023-03-29T14:09:27.286 INFO:tasks.rados.rados.0.smithi017.stdout:1723: writing smithi017107884-19 from 3293084 to 3708737 tid 3 2023-03-29T14:09:27.286 INFO:tasks.rados.rados.0.smithi017.stdout:1724: delete oid 5 current snap is 202 2023-03-29T14:09:27.415 INFO:tasks.rados.rados.0.smithi017.stdout:1724: done (1 left) 2023-03-29T14:09:27.416 INFO:tasks.rados.rados.0.smithi017.stdout:1725: snap_remove snap 199 2023-03-29T14:09:28.179 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:28.179 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:28.179 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:28.275 INFO:tasks.rados.rados.0.smithi017.stdout:1725: done (1 left) 2023-03-29T14:09:28.276 INFO:tasks.rados.rados.0.smithi017.stdout:1726: write (excl) oid 25 current snap is 202 2023-03-29T14:09:28.276 INFO:tasks.rados.rados.0.smithi017.stdout:1723: finishing write tid 1 to smithi017107884-19 2023-03-29T14:09:28.276 INFO:tasks.rados.rados.0.smithi017.stdout:1726: seq_num 832 ranges {783424=489645,2003489=653949,3231577=554550} 2023-03-29T14:09:28.282 INFO:tasks.rados.rados.0.smithi017.stdout:1726: writing smithi017107884-25 from 783424 to 1273069 tid 1 2023-03-29T14:09:28.290 INFO:tasks.rados.rados.0.smithi017.stdout:1726: writing smithi017107884-25 from 2003489 to 2657438 tid 2 2023-03-29T14:09:28.298 INFO:tasks.rados.rados.0.smithi017.stdout:1726: writing smithi017107884-25 from 3231577 to 3786127 tid 3 2023-03-29T14:09:28.299 INFO:tasks.rados.rados.0.smithi017.stdout:1727: read oid 37 snap -1 2023-03-29T14:09:28.299 INFO:tasks.rados.rados.0.smithi017.stdout:1727: expect (ObjNum 829 snap 200 seq_num 829) 2023-03-29T14:09:28.299 INFO:tasks.rados.rados.0.smithi017.stdout:1728: snap_remove snap 198 2023-03-29T14:09:28.316 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:28.311+0000 7ff808915700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:28.319 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:28.318+0000 7ff808915700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:28.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:28.728 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:28.731 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:28.744 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":636,"stamp":"2023-03-29T14:09:28.272155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281535026,"num_objects":968,"num_object_clones":872,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":3893,"num_read_kb":2303043,"num_write":5372,"num_write_kb":1493134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2835,"ondisk_log_size":2835,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1876812,"kb_used_data":1792364,"kb_used_omap":0,"kb_used_meta":84416,"kb_avail":375610548,"statfs":{"total":386547056640,"available":384625201152,"internally_reserved":0,"allocated":1835380736,"data_stored":2272716695,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":86441984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14891464,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":33,"num_read_kb":19123,"num_write":72,"num_write_kb":24042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010121"},"pg_stats":[{"pgid":"3.f","version":"412'129","reported_seq":413,"reported_epoch":419,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:25.291352+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:25.291352+0000","last_peered":"2023-03-29T14:09:25.291352+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:25.291352+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126894,"num_objects":28,"num_object_clones":28,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":65946,"num_write":172,"num_write_kb":51352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":446,"reported_epoch":417,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:23.332908+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:23.332908+0000","last_peered":"2023-03-29T14:09:23.332908+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:23.332908+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"413'260","reported_seq":940,"reported_epoch":419,"state":"active+clean","last_fresh":"2023-03-29T14:09:25.299549+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:25.299549+0000","last_peered":"2023-03-29T14:09:25.299549+0000","last_clean":"2023-03-29T14:09:25.299549+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:25.299549+0000","last_undegraded":"2023-03-29T14:09:25.299549+0000","last_fullsized":"2023-03-29T14:09:25.299549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808848,"num_objects":94,"num_object_clones":84,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":392,"num_read_kb":275565,"num_write":504,"num_write_kb":150092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"406'273","reported_seq":956,"reported_epoch":419,"state":"active+clean","last_fresh":"2023-03-29T14:09:25.298330+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:25.298330+0000","last_peered":"2023-03-29T14:09:25.298330+0000","last_clean":"2023-03-29T14:09:25.298330+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:25.298330+0000","last_undegraded":"2023-03-29T14:09:25.298330+0000","last_fullsized":"2023-03-29T14:09:25.298330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346276,"num_objects":108,"num_object_clones":100,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":231806,"num_write":536,"num_write_kb":151626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"410'222","reported_seq":825,"reported_epoch":417,"state":"active+clean","last_fresh":"2023-03-29T14:09:23.338954+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:23.338954+0000","last_peered":"2023-03-29T14:09:23.338954+0000","last_clean":"2023-03-29T14:09:23.338954+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:23.338954+0000","last_undegraded":"2023-03-29T14:09:23.338954+0000","last_fullsized":"2023-03-29T14:09:23.338954+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23127686,"num_objects":78,"num_object_clones":70,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":320,"num_read_kb":206202,"num_write":444,"num_write_kb":130902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":354,"reported_epoch":419,"state":"active+undersized","last_fresh":"2023-03-29T14:09:25.302599+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:25.302599+0000","last_peered":"2023-03-29T14:09:25.302599+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:25.302599+0000","last_undegraded":"2023-03-29T14:09:25.302599+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"412'156","reported_seq":676,"reported_epoch":419,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:25.300113+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:25.300113+0000","last_peered":"2023-03-29T14:09:25.300113+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:25.300113+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19329040,"num_objects":54,"num_object_clones":48,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":74216,"num_write":286,"num_write_kb":80828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":54}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":574,"reported_epoch":419,"state":"active+clean","last_fresh":"2023-03-29T14:09:25.281288+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:25.281288+0000","last_peered":"2023-03-29T14:09:25.281288+0000","last_clean":"2023-03-29T14:09:25.281288+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:25.281288+0000","last_undegraded":"2023-03-29T14:09:25.281288+0000","last_fullsized":"2023-03-29T14:09:25.281288+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":73865,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":417,"state":"active+undersized","last_fresh":"2023-03-29T14:09:23.368248+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:23.368248+0000","last_peered":"2023-03-29T14:09:23.368248+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:23.368248+0000","last_undegraded":"2023-03-29T14:09:23.368248+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"418'103","reported_seq":328,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:25.465627+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:25.465627+0000","last_peered":"2023-03-29T14:09:25.465627+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:25.465627+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12370612,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":26578,"num_write":130,"num_write_kb":35062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":357,"reported_epoch":419,"state":"active+undersized","last_fresh":"2023-03-29T14:09:25.294450+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:25.294450+0000","last_peered":"2023-03-29T14:09:25.294450+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:25.294450+0000","last_undegraded":"2023-03-29T14:09:25.294450+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"415'113","reported_seq":588,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:23.345965+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:23.345965+0000","last_peered":"2023-03-29T14:09:23.345965+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:23.345965+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15026118,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":107606,"num_write":222,"num_write_kb":66078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":44}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":186,"reported_epoch":417,"state":"active+undersized","last_fresh":"2023-03-29T14:09:23.304983+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:23.304983+0000","last_peered":"2023-03-29T14:09:23.304983+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:23.304983+0000","last_undegraded":"2023-03-29T14:09:23.304983+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"410'155","reported_seq":684,"reported_epoch":417,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:23.421590+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:23.421590+0000","last_peered":"2023-03-29T14:09:23.421590+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:23.421590+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18791150,"num_objects":56,"num_object_clones":50,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":155095,"num_write":298,"num_write_kb":88962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":357,"reported_epoch":419,"state":"active+undersized","last_fresh":"2023-03-29T14:09:25.295146+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:25.295146+0000","last_peered":"2023-03-29T14:09:25.295146+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:25.295146+0000","last_undegraded":"2023-03-29T14:09:25.295146+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":550,"reported_epoch":417,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:23.386285+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:23.386285+0000","last_peered":"2023-03-29T14:09:23.386285+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:23.386285+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"418'208","reported_seq":861,"reported_epoch":419,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:25.387596+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:25.387596+0000","last_peered":"2023-03-29T14:09:25.387596+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:25.387596+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446722,"num_objects":76,"num_object_clones":68,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":370,"num_read_kb":253752,"num_write":404,"num_write_kb":117886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":76}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":357,"reported_epoch":419,"state":"active+undersized","last_fresh":"2023-03-29T14:09:25.289993+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:25.289993+0000","last_peered":"2023-03-29T14:09:25.289993+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:25.289993+0000","last_undegraded":"2023-03-29T14:09:25.289993+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"418'362","reported_seq":1151,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:25.435901+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:25.435901+0000","last_peered":"2023-03-29T14:09:25.435901+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:25.435901+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42658080,"num_objects":128,"num_object_clones":114,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":545,"num_read_kb":335729,"num_write":682,"num_write_kb":200324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":128}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":325,"reported_epoch":417,"state":"active+undersized","last_fresh":"2023-03-29T14:09:23.376595+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:23.376595+0000","last_peered":"2023-03-29T14:09:23.376595+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:23.376595+0000","last_undegraded":"2023-03-29T14:09:23.376595+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":580,"reported_epoch":417,"state":"active+clean","last_fresh":"2023-03-29T14:09:23.347132+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:23.347132+0000","last_peered":"2023-03-29T14:09:23.347132+0000","last_clean":"2023-03-29T14:09:23.347132+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:23.347132+0000","last_undegraded":"2023-03-29T14:09:23.347132+0000","last_fullsized":"2023-03-29T14:09:23.347132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":145,"num_read_kb":79798,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"415'236","reported_seq":819,"reported_epoch":417,"state":"active+clean","last_fresh":"2023-03-29T14:09:24.314240+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:24.314240+0000","last_peered":"2023-03-29T14:09:24.314240+0000","last_clean":"2023-03-29T14:09:24.314240+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:24.314240+0000","last_undegraded":"2023-03-29T14:09:24.314240+0000","last_fullsized":"2023-03-29T14:09:24.314240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4773518,"num_objects":86,"num_object_clones":76,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":293,"num_read_kb":172365,"num_write":422,"num_write_kb":122536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":417,"state":"active+undersized","last_fresh":"2023-03-29T14:09:23.313577+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:23.313577+0000","last_peered":"2023-03-29T14:09:23.313577+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:23.313577+0000","last_undegraded":"2023-03-29T14:09:23.313577+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":357,"reported_epoch":419,"state":"active+undersized","last_fresh":"2023-03-29T14:09:25.293732+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:25.293732+0000","last_peered":"2023-03-29T14:09:25.293732+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:25.293732+0000","last_undegraded":"2023-03-29T14:09:25.293732+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"415'237","reported_seq":860,"reported_epoch":418,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:23.359886+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:23.359886+0000","last_peered":"2023-03-29T14:09:23.359886+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:23.359886+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21850090,"num_objects":86,"num_object_clones":78,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":338,"num_read_kb":213658,"num_write":450,"num_write_kb":134242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":86}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":280354290,"num_objects":964,"num_object_clones":872,"num_object_copies":2892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":16,"num_read":3751,"num_read_kb":2302923,"num_write":5098,"num_write_kb":1489346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2738,"ondisk_log_size":2738,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706104,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":550812,"kb_used_data":527316,"kb_used_omap":0,"kb_used_meta":23488,"kb_avail":93821028,"statfs":{"total":96636764160,"available":96072732672,"internally_reserved":0,"allocated":539971584,"data_stored":671137613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24051712},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706103,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":614240,"kb_used_data":589336,"kb_used_omap":0,"kb_used_meta":24896,"kb_avail":93757600,"statfs":{"total":96636764160,"available":96007782400,"internally_reserved":0,"allocated":603480064,"data_stored":750361465,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25493504},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706103,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":555816,"kb_used_data":531872,"kb_used_omap":0,"kb_used_meta":23936,"kb_avail":93816024,"statfs":{"total":96636764160,"available":96067608576,"internally_reserved":0,"allocated":544636928,"data_stored":677985392,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24510464},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:28.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~259s 2023-03-29T14:09:28.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:28.848+0000 7fba26385700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:28.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:28.852+0000 7fba26385700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:28.918 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:28.919 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1728: done (3 left) 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1729: delete oid 50 current snap is 202 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: finishing write tid 2 to smithi017107884-19 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: finishing write tid 3 to smithi017107884-19 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: finishing write tid 4 to smithi017107884-19 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: finishing write tid 5 to smithi017107884-19 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: oid 19 updating version 0 to 238 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: oid 19 updating version 238 to 242 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: oid 19 version 242 is already newer than 242 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: oid 19 version 242 is already newer than 242 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 242 (ObjNum 830 snap 202 seq_num 830) dirty exists 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1723: left oid 19 (ObjNum 830 snap 202 seq_num 830) 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1726: finishing write tid 1 to smithi017107884-25 2023-03-29T14:09:29.275 INFO:tasks.rados.rados.0.smithi017.stdout:1727: expect (ObjNum 829 snap 200 seq_num 829) 2023-03-29T14:09:29.522 INFO:tasks.rados.rados.0.smithi017.stdout:1726: finishing write tid 2 to smithi017107884-25 2023-03-29T14:09:29.522 INFO:tasks.rados.rados.0.smithi017.stdout:1726: finishing write tid 3 to smithi017107884-25 2023-03-29T14:09:29.522 INFO:tasks.rados.rados.0.smithi017.stdout:1726: finishing write tid 4 to smithi017107884-25 2023-03-29T14:09:29.523 INFO:tasks.rados.rados.0.smithi017.stdout:1723: done (3 left) 2023-03-29T14:09:29.523 INFO:tasks.rados.rados.0.smithi017.stdout:1727: done (2 left) 2023-03-29T14:09:29.523 INFO:tasks.rados.rados.0.smithi017.stdout:1729: done (1 left) 2023-03-29T14:09:29.523 INFO:tasks.rados.rados.0.smithi017.stdout:1730: snap_remove snap 196 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1730: done (1 left) 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1731: read oid 27 snap -1 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1731: expect deleted 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1726: finishing write tid 5 to smithi017107884-25 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1726: oid 25 updating version 0 to 278 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1726: oid 25 version 278 is already newer than 274 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1726: oid 25 version 278 is already newer than 278 2023-03-29T14:09:30.281 INFO:tasks.rados.rados.0.smithi017.stdout:1726: oid 25 version 278 is already newer than 278 2023-03-29T14:09:30.282 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 278 (ObjNum 832 snap 202 seq_num 832) dirty exists 2023-03-29T14:09:30.282 INFO:tasks.rados.rados.0.smithi017.stdout:1726: left oid 25 (ObjNum 832 snap 202 seq_num 832) 2023-03-29T14:09:30.282 INFO:tasks.rados.rados.0.smithi017.stdout:1732: read oid 17 snap -1 2023-03-29T14:09:30.282 INFO:tasks.rados.rados.0.smithi017.stdout:1732: expect (ObjNum 764 snap 188 seq_num 764) 2023-03-29T14:09:30.282 INFO:tasks.rados.rados.0.smithi017.stdout:1726: done (2 left) 2023-03-29T14:09:30.282 INFO:tasks.rados.rados.0.smithi017.stdout:1733: write (excl) oid 6 current snap is 202 2023-03-29T14:09:30.282 INFO:tasks.rados.rados.0.smithi017.stdout:1733: seq_num 834 ranges {417038=640143,1720453=296585} 2023-03-29T14:09:30.286 INFO:tasks.rados.rados.0.smithi017.stdout:1733: writing smithi017107884-6 from 417038 to 1057181 tid 1 2023-03-29T14:09:30.292 INFO:tasks.rados.rados.0.smithi017.stdout:1733: writing smithi017107884-6 from 1720453 to 2017038 tid 2 2023-03-29T14:09:30.292 INFO:tasks.rados.rados.0.smithi017.stdout:1734: snap_remove snap 191 2023-03-29T14:09:30.920 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:31.049 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:31.047+0000 7fe9424c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:31.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:31.051+0000 7fe9424c9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:31.117 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:31.118 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:31.284 INFO:tasks.rados.rados.0.smithi017.stdout:1734: done (3 left) 2023-03-29T14:09:31.284 INFO:tasks.rados.rados.0.smithi017.stdout:1735: write oid 15 current snap is 202 2023-03-29T14:09:31.285 INFO:tasks.rados.rados.0.smithi017.stdout:1735: seq_num 835 ranges {433845=715392,1662582=371263} 2023-03-29T14:09:31.290 INFO:tasks.rados.rados.0.smithi017.stdout:1735: writing smithi017107884-15 from 433845 to 1149237 tid 1 2023-03-29T14:09:31.295 INFO:tasks.rados.rados.0.smithi017.stdout:1735: writing smithi017107884-15 from 1662582 to 2033845 tid 2 2023-03-29T14:09:31.296 INFO:tasks.rados.rados.0.smithi017.stdout:1736: write oid 9 current snap is 202 2023-03-29T14:09:31.296 INFO:tasks.rados.rados.0.smithi017.stdout:1736: seq_num 836 ranges {450652=790641,1721064=329588} 2023-03-29T14:09:31.302 INFO:tasks.rados.rados.0.smithi017.stdout:1736: writing smithi017107884-9 from 450652 to 1241293 tid 1 2023-03-29T14:09:31.307 INFO:tasks.rados.rados.0.smithi017.stdout:1736: writing smithi017107884-9 from 1721064 to 2050652 tid 2 2023-03-29T14:09:31.307 INFO:tasks.rados.rados.0.smithi017.stdout:1737: write (excl) oid 21 current snap is 202 2023-03-29T14:09:31.307 INFO:tasks.rados.rados.0.smithi017.stdout:1737: seq_num 837 ranges {467459=582243,1495899=404945} 2023-03-29T14:09:31.312 INFO:tasks.rados.rados.0.smithi017.stdout:1737: writing smithi017107884-21 from 467459 to 1049702 tid 1 2023-03-29T14:09:31.317 INFO:tasks.rados.rados.0.smithi017.stdout:1737: writing smithi017107884-21 from 1495899 to 1900844 tid 2 2023-03-29T14:09:31.318 INFO:tasks.rados.rados.0.smithi017.stdout:1738: write (excl) oid 47 current snap is 202 2023-03-29T14:09:31.318 INFO:tasks.rados.rados.0.smithi017.stdout:1738: seq_num 838 ranges {484266=657492,1554381=529885} 2023-03-29T14:09:31.325 INFO:tasks.rados.rados.0.smithi017.stdout:1738: writing smithi017107884-47 from 484266 to 1141758 tid 1 2023-03-29T14:09:31.332 INFO:tasks.rados.rados.0.smithi017.stdout:1738: writing smithi017107884-47 from 1554381 to 2084266 tid 2 2023-03-29T14:09:31.332 INFO:tasks.rados.rados.0.smithi017.stdout:1731: done (6 left) 2023-03-29T14:09:31.332 INFO:tasks.rados.rados.0.smithi017.stdout:1739: write (excl) oid 42 current snap is 202 2023-03-29T14:09:31.332 INFO:tasks.rados.rados.0.smithi017.stdout:1732: expect (ObjNum 764 snap 188 seq_num 764) 2023-03-29T14:09:31.591 INFO:tasks.rados.rados.0.smithi017.stdout:1733: finishing write tid 1 to smithi017107884-6 2023-03-29T14:09:31.591 INFO:tasks.rados.rados.0.smithi017.stdout:1733: finishing write tid 2 to smithi017107884-6 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1733: finishing write tid 3 to smithi017107884-6 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1733: finishing write tid 4 to smithi017107884-6 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1733: oid 6 updating version 0 to 212 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1733: oid 6 version 212 is already newer than 212 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1733: oid 6 version 212 is already newer than 209 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 6 v 212 (ObjNum 834 snap 202 seq_num 834) dirty exists 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1733: left oid 6 (ObjNum 834 snap 202 seq_num 834) 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1736: finishing write tid 1 to smithi017107884-9 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1735: finishing write tid 1 to smithi017107884-15 2023-03-29T14:09:31.592 INFO:tasks.rados.rados.0.smithi017.stdout:1739: seq_num 839 ranges {501073=732741,1896510=204563} 2023-03-29T14:09:31.598 INFO:tasks.rados.rados.0.smithi017.stdout:1739: writing smithi017107884-42 from 501073 to 1233814 tid 1 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1739: writing smithi017107884-42 from 1896510 to 2101073 tid 2 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1732: done (6 left) 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1733: done (5 left) 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1740: snap_create 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1735: finishing write tid 2 to smithi017107884-15 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1735: finishing write tid 3 to smithi017107884-15 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1735: finishing write tid 4 to smithi017107884-15 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1735: oid 15 updating version 0 to 366 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1735: oid 15 version 366 is already newer than 366 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1735: oid 15 version 366 is already newer than 366 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 15 v 366 (ObjNum 835 snap 202 seq_num 835) dirty exists 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1735: left oid 15 (ObjNum 835 snap 202 seq_num 835) 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1737: finishing write tid 1 to smithi017107884-21 2023-03-29T14:09:31.604 INFO:tasks.rados.rados.0.smithi017.stdout:1736: finishing write tid 2 to smithi017107884-9 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1736: finishing write tid 3 to smithi017107884-9 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1736: finishing write tid 4 to smithi017107884-9 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1736: oid 9 updating version 0 to 282 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1736: oid 9 version 282 is already newer than 282 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1736: oid 9 version 282 is already newer than 282 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 282 (ObjNum 836 snap 202 seq_num 836) dirty exists 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1736: left oid 9 (ObjNum 836 snap 202 seq_num 836) 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1738: finishing write tid 1 to smithi017107884-47 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1737: finishing write tid 2 to smithi017107884-21 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1737: finishing write tid 3 to smithi017107884-21 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1737: finishing write tid 4 to smithi017107884-21 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1737: oid 21 updating version 0 to 213 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1737: oid 21 updating version 213 to 216 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1737: oid 21 version 216 is already newer than 216 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 216 (ObjNum 837 snap 202 seq_num 837) dirty exists 2023-03-29T14:09:31.605 INFO:tasks.rados.rados.0.smithi017.stdout:1737: left oid 21 (ObjNum 837 snap 202 seq_num 837) 2023-03-29T14:09:31.649 INFO:tasks.rados.rados.0.smithi017.stdout:1738: finishing write tid 2 to smithi017107884-47 2023-03-29T14:09:31.650 INFO:tasks.rados.rados.0.smithi017.stdout:1738: finishing write tid 3 to smithi017107884-47 2023-03-29T14:09:31.652 INFO:tasks.rados.rados.0.smithi017.stdout:1738: finishing write tid 4 to smithi017107884-47 2023-03-29T14:09:31.652 INFO:tasks.rados.rados.0.smithi017.stdout:1738: oid 47 updating version 0 to 241 2023-03-29T14:09:31.653 INFO:tasks.rados.rados.0.smithi017.stdout:1738: oid 47 version 241 is already newer than 239 2023-03-29T14:09:31.653 INFO:tasks.rados.rados.0.smithi017.stdout:1738: oid 47 version 241 is already newer than 241 2023-03-29T14:09:31.653 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 241 (ObjNum 838 snap 202 seq_num 838) dirty exists 2023-03-29T14:09:31.653 INFO:tasks.rados.rados.0.smithi017.stdout:1738: left oid 47 (ObjNum 838 snap 202 seq_num 838) 2023-03-29T14:09:31.656 INFO:tasks.rados.rados.0.smithi017.stdout:1739: finishing write tid 1 to smithi017107884-42 2023-03-29T14:09:31.708 INFO:tasks.rados.rados.0.smithi017.stdout:1739: finishing write tid 2 to smithi017107884-42 2023-03-29T14:09:31.709 INFO:tasks.rados.rados.0.smithi017.stdout:1739: finishing write tid 3 to smithi017107884-42 2023-03-29T14:09:31.709 INFO:tasks.rados.rados.0.smithi017.stdout:1739: finishing write tid 4 to smithi017107884-42 2023-03-29T14:09:31.710 INFO:tasks.rados.rados.0.smithi017.stdout:1739: oid 42 updating version 0 to 286 2023-03-29T14:09:31.710 INFO:tasks.rados.rados.0.smithi017.stdout:1739: oid 42 version 286 is already newer than 283 2023-03-29T14:09:31.710 INFO:tasks.rados.rados.0.smithi017.stdout:1739: oid 42 version 286 is already newer than 286 2023-03-29T14:09:31.710 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 286 (ObjNum 839 snap 202 seq_num 839) dirty exists 2023-03-29T14:09:31.710 INFO:tasks.rados.rados.0.smithi017.stdout:1739: left oid 42 (ObjNum 839 snap 202 seq_num 839) 2023-03-29T14:09:32.288 INFO:tasks.rados.rados.0.smithi017.stdout:1735: done (5 left) 2023-03-29T14:09:32.288 INFO:tasks.rados.rados.0.smithi017.stdout:1736: done (4 left) 2023-03-29T14:09:32.288 INFO:tasks.rados.rados.0.smithi017.stdout:1737: done (3 left) 2023-03-29T14:09:32.288 INFO:tasks.rados.rados.0.smithi017.stdout:1738: done (2 left) 2023-03-29T14:09:32.288 INFO:tasks.rados.rados.0.smithi017.stdout:1739: done (1 left) 2023-03-29T14:09:32.288 INFO:tasks.rados.rados.0.smithi017.stdout:1740: done (0 left) 2023-03-29T14:09:32.289 INFO:tasks.rados.rados.0.smithi017.stdout:1741: write (excl) oid 3 current snap is 203 2023-03-29T14:09:32.289 INFO:tasks.rados.rados.0.smithi017.stdout:1741: seq_num 840 ranges {517880=407990,1554992=562888} 2023-03-29T14:09:32.292 INFO:tasks.rados.rados.0.smithi017.stdout:1741: writing smithi017107884-3 from 517880 to 925870 tid 1 2023-03-29T14:09:32.300 INFO:tasks.rados.rados.0.smithi017.stdout:1741: writing smithi017107884-3 from 1554992 to 2117880 tid 2 2023-03-29T14:09:32.300 INFO:tasks.rados.rados.0.smithi017.stdout:1742: delete oid 15 current snap is 203 2023-03-29T14:09:32.382 INFO:tasks.rados.rados.0.smithi017.stdout:1741: finishing write tid 1 to smithi017107884-3 2023-03-29T14:09:32.403 INFO:tasks.rados.rados.0.smithi017.stdout:1741: finishing write tid 2 to smithi017107884-3 2023-03-29T14:09:32.449 INFO:tasks.rados.rados.0.smithi017.stdout:1741: finishing write tid 3 to smithi017107884-3 2023-03-29T14:09:32.455 INFO:tasks.rados.rados.0.smithi017.stdout:1741: finishing write tid 4 to smithi017107884-3 2023-03-29T14:09:32.455 INFO:tasks.rados.rados.0.smithi017.stdout:1741: oid 3 updating version 0 to 370 2023-03-29T14:09:32.455 INFO:tasks.rados.rados.0.smithi017.stdout:1741: oid 3 version 370 is already newer than 367 2023-03-29T14:09:32.455 INFO:tasks.rados.rados.0.smithi017.stdout:1741: oid 3 version 370 is already newer than 370 2023-03-29T14:09:32.456 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 370 (ObjNum 840 snap 203 seq_num 840) dirty exists 2023-03-29T14:09:32.456 INFO:tasks.rados.rados.0.smithi017.stdout:1741: left oid 3 (ObjNum 840 snap 203 seq_num 840) 2023-03-29T14:09:32.502 INFO:tasks.rados.rados.0.smithi017.stdout:1741: done (1 left) 2023-03-29T14:09:32.502 INFO:tasks.rados.rados.0.smithi017.stdout:1742: done (0 left) 2023-03-29T14:09:32.502 INFO:tasks.rados.rados.0.smithi017.stdout:1743: write oid 49 current snap is 203 2023-03-29T14:09:32.502 INFO:tasks.rados.rados.0.smithi017.stdout:1743: seq_num 842 ranges {551494=558488,1671956=479538} 2023-03-29T14:09:32.508 INFO:tasks.rados.rados.0.smithi017.stdout:1743: writing smithi017107884-49 from 551494 to 1109982 tid 1 2023-03-29T14:09:32.513 INFO:tasks.rados.rados.0.smithi017.stdout:1743: writing smithi017107884-49 from 1671956 to 2151494 tid 2 2023-03-29T14:09:32.514 INFO:tasks.rados.rados.0.smithi017.stdout:1744: delete oid 10 current snap is 203 2023-03-29T14:09:32.578 INFO:tasks.rados.rados.0.smithi017.stdout:1743: finishing write tid 1 to smithi017107884-49 2023-03-29T14:09:32.579 INFO:tasks.rados.rados.0.smithi017.stdout:1744: done (1 left) 2023-03-29T14:09:32.579 INFO:tasks.rados.rados.0.smithi017.stdout:1745: snap_remove snap 195 2023-03-29T14:09:33.118 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:33.247 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:33.245+0000 7f466d087700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:33.250 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:33.249+0000 7f466d087700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:33.262 INFO:tasks.rados.rados.0.smithi017.stdout:1745: done (1 left) 2023-03-29T14:09:33.262 INFO:tasks.rados.rados.0.smithi017.stdout:1746: read oid 32 snap -1 2023-03-29T14:09:33.262 INFO:tasks.rados.rados.0.smithi017.stdout:1746: expect (ObjNum 667 snap 163 seq_num 667) 2023-03-29T14:09:33.262 INFO:tasks.rados.rados.0.smithi017.stdout:1747: write (excl) oid 9 current snap is 203 2023-03-29T14:09:33.262 INFO:tasks.rados.rados.0.smithi017.stdout:1743: finishing write tid 2 to smithi017107884-49 2023-03-29T14:09:33.262 INFO:tasks.rados.rados.0.smithi017.stdout:1747: seq_num 844 ranges {585108=425339,1788920=396188} 2023-03-29T14:09:33.267 INFO:tasks.rados.rados.0.smithi017.stdout:1747: writing smithi017107884-9 from 585108 to 1010447 tid 1 2023-03-29T14:09:33.274 INFO:tasks.rados.rados.0.smithi017.stdout:1747: writing smithi017107884-9 from 1788920 to 2185108 tid 2 2023-03-29T14:09:33.274 INFO:tasks.rados.rados.0.smithi017.stdout:1748: delete oid 47 current snap is 203 2023-03-29T14:09:33.274 INFO:tasks.rados.rados.0.smithi017.stdout:1743: finishing write tid 3 to smithi017107884-49 2023-03-29T14:09:33.274 INFO:tasks.rados.rados.0.smithi017.stdout:1743: finishing write tid 4 to smithi017107884-49 2023-03-29T14:09:33.274 INFO:tasks.rados.rados.0.smithi017.stdout:1743: oid 49 updating version 0 to 376 2023-03-29T14:09:33.274 INFO:tasks.rados.rados.0.smithi017.stdout:1743: oid 49 version 376 is already newer than 375 2023-03-29T14:09:33.274 INFO:tasks.rados.rados.0.smithi017.stdout:1743: oid 49 version 376 is already newer than 376 2023-03-29T14:09:33.275 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 376 (ObjNum 842 snap 203 seq_num 842) dirty exists 2023-03-29T14:09:33.275 INFO:tasks.rados.rados.0.smithi017.stdout:1743: left oid 49 (ObjNum 842 snap 203 seq_num 842) 2023-03-29T14:09:33.389 INFO:tasks.rados.rados.0.smithi017.stdout:1746: expect (ObjNum 667 snap 163 seq_num 667) 2023-03-29T14:09:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:33.471 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:33.472 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: finishing write tid 1 to smithi017107884-9 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: finishing write tid 2 to smithi017107884-9 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: finishing write tid 3 to smithi017107884-9 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: finishing write tid 4 to smithi017107884-9 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: oid 9 updating version 0 to 290 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: oid 9 version 290 is already newer than 290 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: oid 9 version 290 is already newer than 287 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 290 (ObjNum 844 snap 203 seq_num 844) dirty exists 2023-03-29T14:09:33.596 INFO:tasks.rados.rados.0.smithi017.stdout:1747: left oid 9 (ObjNum 844 snap 203 seq_num 844) 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1743: done (3 left) 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1746: done (2 left) 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1747: done (1 left) 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1748: done (0 left) 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1749: read oid 35 snap 197 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1749: expect (ObjNum 786 snap 191 seq_num 786) 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1750: write (excl) oid 43 current snap is 203 2023-03-29T14:09:33.597 INFO:tasks.rados.rados.0.smithi017.stdout:1750: seq_num 846 ranges {618722=575837,1905884=312838} 2023-03-29T14:09:33.602 INFO:tasks.rados.rados.0.smithi017.stdout:1750: writing smithi017107884-43 from 618722 to 1194559 tid 1 2023-03-29T14:09:33.608 INFO:tasks.rados.rados.0.smithi017.stdout:1750: writing smithi017107884-43 from 1905884 to 2218722 tid 2 2023-03-29T14:09:33.608 INFO:tasks.rados.rados.0.smithi017.stdout:1751: read oid 20 snap -1 2023-03-29T14:09:33.608 INFO:tasks.rados.rados.0.smithi017.stdout:1751: expect (ObjNum 804 snap 196 seq_num 804) 2023-03-29T14:09:33.608 INFO:tasks.rados.rados.0.smithi017.stdout:1752: write oid 24 current snap is 203 2023-03-29T14:09:33.608 INFO:tasks.rados.rados.0.smithi017.stdout:1752: seq_num 847 ranges {635529=651086,1848013=387516} 2023-03-29T14:09:33.614 INFO:tasks.rados.rados.0.smithi017.stdout:1752: writing smithi017107884-24 from 635529 to 1286615 tid 1 2023-03-29T14:09:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:1752: writing smithi017107884-24 from 1848013 to 2235529 tid 2 2023-03-29T14:09:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:1753: write oid 47 current snap is 203 2023-03-29T14:09:33.620 INFO:tasks.rados.rados.0.smithi017.stdout:1753: seq_num 848 ranges {652336=726335,1906495=345841} 2023-03-29T14:09:33.627 INFO:tasks.rados.rados.0.smithi017.stdout:1753: writing smithi017107884-47 from 652336 to 1378671 tid 1 2023-03-29T14:09:33.632 INFO:tasks.rados.rados.0.smithi017.stdout:1753: writing smithi017107884-47 from 1906495 to 2252336 tid 2 2023-03-29T14:09:33.632 INFO:tasks.rados.rados.0.smithi017.stdout:1754: snap_remove snap 202 2023-03-29T14:09:33.747 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:33.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-03-29T14:09:33.886 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:33.879+0000 7f9185780700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:33.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:33.888+0000 7f9185780700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:34.259 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~264s 2023-03-29T14:09:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:1754: done (5 left) 2023-03-29T14:09:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:1755: read oid 28 snap -1 2023-03-29T14:09:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:1755: expect (ObjNum 810 snap 196 seq_num 810) 2023-03-29T14:09:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:1756: write oid 14 current snap is 203 2023-03-29T14:09:34.292 INFO:tasks.rados.rados.0.smithi017.stdout:1756: seq_num 849 ranges {669143=401584,1564977=426959} 2023-03-29T14:09:34.295 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2023-03-29T14:09:34.298 INFO:tasks.rados.rados.0.smithi017.stdout:1756: writing smithi017107884-14 from 669143 to 1070727 tid 1 2023-03-29T14:09:34.303 INFO:tasks.rados.rados.0.smithi017.stdout:1756: writing smithi017107884-14 from 1564977 to 1991936 tid 2 2023-03-29T14:09:34.303 INFO:tasks.rados.rados.0.smithi017.stdout:1757: read oid 44 snap 201 2023-03-29T14:09:34.303 INFO:tasks.rados.rados.0.smithi017.stdout:1757: expect deleted 2023-03-29T14:09:34.303 INFO:tasks.rados.rados.0.smithi017.stdout:1758: delete oid 45 current snap is 203 2023-03-29T14:09:34.303 INFO:tasks.rados.rados.0.smithi017.stdout:1749: expect (ObjNum 786 snap 191 seq_num 786) 2023-03-29T14:09:34.307 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-03-29T14:09:34.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:34.439+0000 7f358d1be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:34.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:34.449+0000 7f358d1be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:34.641 INFO:tasks.rados.rados.0.smithi017.stdout:1750: finishing write tid 1 to smithi017107884-43 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: finishing write tid 1 to smithi017107884-24 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: finishing write tid 2 to smithi017107884-24 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: finishing write tid 3 to smithi017107884-24 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: finishing write tid 4 to smithi017107884-24 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: oid 24 updating version 0 to 117 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: oid 24 version 117 is already newer than 117 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: oid 24 version 117 is already newer than 117 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 24 v 117 (ObjNum 847 snap 203 seq_num 847) dirty exists 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1752: left oid 24 (ObjNum 847 snap 203 seq_num 847) 2023-03-29T14:09:34.642 INFO:tasks.rados.rados.0.smithi017.stdout:1751: expect (ObjNum 804 snap 196 seq_num 804) 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1750: finishing write tid 2 to smithi017107884-43 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1750: finishing write tid 3 to smithi017107884-43 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1749: done (8 left) 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1751: done (7 left) 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1752: done (6 left) 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1758: done (5 left) 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1759: snap_create 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1750: finishing write tid 4 to smithi017107884-43 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1750: oid 43 updating version 0 to 161 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1750: oid 43 version 161 is already newer than 161 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1750: oid 43 version 161 is already newer than 158 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 43 v 161 (ObjNum 846 snap 203 seq_num 846) dirty exists 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1750: left oid 43 (ObjNum 846 snap 203 seq_num 846) 2023-03-29T14:09:34.971 INFO:tasks.rados.rados.0.smithi017.stdout:1753: finishing write tid 1 to smithi017107884-47 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1753: finishing write tid 2 to smithi017107884-47 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1753: finishing write tid 3 to smithi017107884-47 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1753: finishing write tid 4 to smithi017107884-47 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1753: oid 47 updating version 0 to 246 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1753: oid 47 version 246 is already newer than 246 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1753: oid 47 version 246 is already newer than 246 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 47 v 246 (ObjNum 848 snap 203 seq_num 848) dirty exists 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1753: left oid 47 (ObjNum 848 snap 203 seq_num 848) 2023-03-29T14:09:34.972 INFO:tasks.rados.rados.0.smithi017.stdout:1755: expect (ObjNum 810 snap 196 seq_num 810) 2023-03-29T14:09:35.161 INFO:tasks.rados.rados.0.smithi017.stdout:1756: finishing write tid 1 to smithi017107884-14 2023-03-29T14:09:35.161 INFO:tasks.rados.rados.0.smithi017.stdout:1756: finishing write tid 2 to smithi017107884-14 2023-03-29T14:09:35.161 INFO:tasks.rados.rados.0.smithi017.stdout:1756: finishing write tid 3 to smithi017107884-14 2023-03-29T14:09:35.161 INFO:tasks.rados.rados.0.smithi017.stdout:1756: finishing write tid 4 to smithi017107884-14 2023-03-29T14:09:35.162 INFO:tasks.rados.rados.0.smithi017.stdout:1756: oid 14 updating version 0 to 160 2023-03-29T14:09:35.162 INFO:tasks.rados.rados.0.smithi017.stdout:1756: oid 14 version 160 is already newer than 160 2023-03-29T14:09:35.162 INFO:tasks.rados.rados.0.smithi017.stdout:1756: oid 14 version 160 is already newer than 160 2023-03-29T14:09:35.162 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 160 (ObjNum 849 snap 203 seq_num 849) dirty exists 2023-03-29T14:09:35.162 INFO:tasks.rados.rados.0.smithi017.stdout:1756: left oid 14 (ObjNum 849 snap 203 seq_num 849) 2023-03-29T14:09:35.293 INFO:tasks.rados.rados.0.smithi017.stdout:1750: done (5 left) 2023-03-29T14:09:35.294 INFO:tasks.rados.rados.0.smithi017.stdout:1753: done (4 left) 2023-03-29T14:09:35.294 INFO:tasks.rados.rados.0.smithi017.stdout:1755: done (3 left) 2023-03-29T14:09:35.294 INFO:tasks.rados.rados.0.smithi017.stdout:1756: done (2 left) 2023-03-29T14:09:35.294 INFO:tasks.rados.rados.0.smithi017.stdout:1757: done (1 left) 2023-03-29T14:09:35.294 INFO:tasks.rados.rados.0.smithi017.stdout:1759: done (0 left) 2023-03-29T14:09:35.294 INFO:tasks.rados.rados.0.smithi017.stdout:1760: snap_remove snap 203 2023-03-29T14:09:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2023-03-29T14:09:35.309 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-03-29T14:09:35.443 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:35.436+0000 7fd079126700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:35.445 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:35.444+0000 7fd079126700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:35.473 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:35.603 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:35.600+0000 7fbe497e3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:35.606 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:35.605+0000 7fbe497e3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:35.675 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:35.677 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:35.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:36.298 INFO:tasks.rados.rados.0.smithi017.stdout:1760: done (0 left) 2023-03-29T14:09:36.298 INFO:tasks.rados.rados.0.smithi017.stdout:1761: snap_remove snap 201 2023-03-29T14:09:36.301 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2023-03-29T14:09:36.313 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-03-29T14:09:36.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:36.440+0000 7f219bbb3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:36.449 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:36.448+0000 7f219bbb3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:37.302 INFO:tasks.rados.rados.0.smithi017.stdout:1761: done (0 left) 2023-03-29T14:09:37.302 INFO:tasks.rados.rados.0.smithi017.stdout:1762: read oid 47 snap -1 2023-03-29T14:09:37.302 INFO:tasks.rados.rados.0.smithi017.stdout:1762: expect (ObjNum 848 snap 203 seq_num 848) 2023-03-29T14:09:37.302 INFO:tasks.rados.rados.0.smithi017.stdout:1763: delete oid 25 current snap is 204 2023-03-29T14:09:37.304 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2023-03-29T14:09:37.316 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-03-29T14:09:37.317 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:37.445 INFO:tasks.rados.rados.0.smithi017.stdout:1762: expect (ObjNum 848 snap 203 seq_num 848) 2023-03-29T14:09:37.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:37.447+0000 7fa782f58700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:37.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:37.453+0000 7fa782f58700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:37.534 INFO:tasks.rados.rados.0.smithi017.stdout:1762: done (1 left) 2023-03-29T14:09:37.534 INFO:tasks.rados.rados.0.smithi017.stdout:1763: done (0 left) 2023-03-29T14:09:37.534 INFO:tasks.rados.rados.0.smithi017.stdout:1764: snap_remove snap 200 2023-03-29T14:09:37.659 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:37.673 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2023-03-29T14:09:37.300240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241231576,"num_objects":1004,"num_object_clones":908,"num_object_copies":3012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3974,"num_read_kb":2353790,"num_write":5496,"num_write_kb":1525230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2907,"ondisk_log_size":2907,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1917008,"kb_used_data":1831600,"kb_used_omap":0,"kb_used_meta":85376,"kb_avail":375570352,"statfs":{"total":386547056640,"available":384584040448,"internally_reserved":0,"allocated":1875558400,"data_stored":2331587792,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":87425024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30656034,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":54,"num_read_kb":36263,"num_write":86,"num_write_kb":20088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039772"},"pg_stats":[{"pgid":"3.f","version":"412'129","reported_seq":420,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.283292+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:33.283292+0000","last_peered":"2023-03-29T14:09:33.283292+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:33.283292+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126894,"num_objects":28,"num_object_clones":28,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":65946,"num_write":172,"num_write_kb":51352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":454,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.347045+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:34.347045+0000","last_peered":"2023-03-29T14:09:34.347045+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:34.347045+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"425'262","reported_seq":952,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:33.273429+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:33.273429+0000","last_peered":"2023-03-29T14:09:33.273429+0000","last_clean":"2023-03-29T14:09:33.273429+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:33.273429+0000","last_undegraded":"2023-03-29T14:09:33.273429+0000","last_fullsized":"2023-03-29T14:09:33.273429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25951262,"num_objects":96,"num_object_clones":86,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":399,"num_read_kb":282257,"num_write":504,"num_write_kb":150092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"426'290","reported_seq":993,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:33.478203+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:33.478203+0000","last_peered":"2023-03-29T14:09:33.478203+0000","last_clean":"2023-03-29T14:09:33.478203+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:33.478203+0000","last_undegraded":"2023-03-29T14:09:33.478203+0000","last_fullsized":"2023-03-29T14:09:33.478203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22287534,"num_objects":116,"num_object_clones":108,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":394,"num_read_kb":231810,"num_write":570,"num_write_kb":160574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"427'224","reported_seq":838,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.411528+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:34.411528+0000","last_peered":"2023-03-29T14:09:34.411528+0000","last_clean":"2023-03-29T14:09:34.411528+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:34.411528+0000","last_undegraded":"2023-03-29T14:09:34.411528+0000","last_fullsized":"2023-03-29T14:09:34.411528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17345748,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":327,"num_read_kb":216072,"num_write":444,"num_write_kb":130902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":361,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.278883+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:33.278883+0000","last_peered":"2023-03-29T14:09:33.278883+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:33.278883+0000","last_undegraded":"2023-03-29T14:09:33.278883+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"427'160","reported_seq":690,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.344758+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:34.344758+0000","last_peered":"2023-03-29T14:09:34.344758+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:34.344758+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18992966,"num_objects":56,"num_object_clones":50,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":74217,"num_write":294,"num_write_kb":82448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":56}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":584,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.344693+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:34.344693+0000","last_peered":"2023-03-29T14:09:34.344693+0000","last_clean":"2023-03-29T14:09:34.344693+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:34.344693+0000","last_undegraded":"2023-03-29T14:09:34.344693+0000","last_fullsized":"2023-03-29T14:09:34.344693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":82092,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.370260+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:34.370260+0000","last_peered":"2023-03-29T14:09:34.370260+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:34.370260+0000","last_undegraded":"2023-03-29T14:09:34.370260+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"418'103","reported_seq":339,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.325171+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:33.325171+0000","last_peered":"2023-03-29T14:09:33.325171+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:33.325171+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12370612,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":34367,"num_write":130,"num_write_kb":35062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.276742+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:33.276742+0000","last_peered":"2023-03-29T14:09:33.276742+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:33.276742+0000","last_undegraded":"2023-03-29T14:09:33.276742+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"426'117","reported_seq":603,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.763162+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:33.763162+0000","last_peered":"2023-03-29T14:09:33.763162+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:33.763162+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12135380,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":167,"num_read_kb":107607,"num_write":230,"num_write_kb":68108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":194,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.381357+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:34.381357+0000","last_peered":"2023-03-29T14:09:34.381357+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:34.381357+0000","last_undegraded":"2023-03-29T14:09:34.381357+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":701,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.364381+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:34.364381+0000","last_peered":"2023-03-29T14:09:34.364381+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:34.364381+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.285195+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:33.285195+0000","last_peered":"2023-03-29T14:09:33.285195+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:33.285195+0000","last_undegraded":"2023-03-29T14:09:33.285195+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":558,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.345635+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:34.345635+0000","last_peered":"2023-03-29T14:09:34.345635+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:34.345635+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"424'216","reported_seq":882,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.277557+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:33.277557+0000","last_peered":"2023-03-29T14:09:33.277557+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:33.277557+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900544,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":376,"num_read_kb":253754,"num_write":420,"num_write_kb":121648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":80}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.284373+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:33.284373+0000","last_peered":"2023-03-29T14:09:33.284373+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:33.284373+0000","last_undegraded":"2023-03-29T14:09:33.284373+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"425'376","reported_seq":1185,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.386480+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:33.386480+0000","last_peered":"2023-03-29T14:09:33.386480+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:33.386480+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29380352,"num_objects":136,"num_object_clones":122,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":559,"num_read_kb":345140,"num_write":706,"num_write_kb":206376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":136}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.344282+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:34.344282+0000","last_peered":"2023-03-29T14:09:34.344282+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:34.344282+0000","last_undegraded":"2023-03-29T14:09:34.344282+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":591,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.353358+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:34.353358+0000","last_peered":"2023-03-29T14:09:34.353358+0000","last_clean":"2023-03-29T14:09:34.353358+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:34.353358+0000","last_undegraded":"2023-03-29T14:09:34.353358+0000","last_fullsized":"2023-03-29T14:09:34.353358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":88544,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"426'246","reported_seq":844,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.368552+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:34.368552+0000","last_peered":"2023-03-29T14:09:34.368552+0000","last_clean":"2023-03-29T14:09:34.368552+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:34.368552+0000","last_undegraded":"2023-03-29T14:09:34.368552+0000","last_fullsized":"2023-03-29T14:09:34.368552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278190,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":299,"num_read_kb":172367,"num_write":438,"num_write_kb":126954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.362370+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:34.362370+0000","last_peered":"2023-03-29T14:09:34.362370+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:34.362370+0000","last_undegraded":"2023-03-29T14:09:34.362370+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.279707+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:33.279707+0000","last_peered":"2023-03-29T14:09:33.279707+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:33.279707+0000","last_undegraded":"2023-03-29T14:09:33.279707+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"420'242","reported_seq":877,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.322307+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:33.322307+0000","last_peered":"2023-03-29T14:09:33.322307+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:33.322307+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22051774,"num_objects":88,"num_object_clones":80,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":341,"num_read_kb":213659,"num_write":460,"num_write_kb":137770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":88}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":240050840,"num_objects":1000,"num_object_clones":908,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3832,"num_read_kb":2353670,"num_write":5222,"num_write_kb":1521442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2810,"ondisk_log_size":2810,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":563676,"kb_used_data":539860,"kb_used_omap":0,"kb_used_meta":23808,"kb_avail":93808164,"statfs":{"total":96636764160,"available":96059559936,"internally_reserved":0,"allocated":552816640,"data_stored":689079797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24379392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706113,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":628068,"kb_used_data":602844,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":93743772,"statfs":{"total":96636764160,"available":95993622528,"internally_reserved":0,"allocated":617312256,"data_stored":770470945,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706113,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":569320,"kb_used_data":545056,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":93802520,"statfs":{"total":96636764160,"available":96053780480,"internally_reserved":0,"allocated":558137344,"data_stored":698804825,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:37.676 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:37.678 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:37.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:37.806+0000 7fe8338e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:37.813+0000 7fe8338e7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:37.853+0000 7f67b181d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:37.859 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:37.858+0000 7f67b181d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:37.922 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:37.925 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:37.926 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:38.024 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:38.026 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:38.038 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2023-03-29T14:09:37.300240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241231576,"num_objects":1004,"num_object_clones":908,"num_object_copies":3012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3974,"num_read_kb":2353790,"num_write":5496,"num_write_kb":1525230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2907,"ondisk_log_size":2907,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1917008,"kb_used_data":1831600,"kb_used_omap":0,"kb_used_meta":85376,"kb_avail":375570352,"statfs":{"total":386547056640,"available":384584040448,"internally_reserved":0,"allocated":1875558400,"data_stored":2331587792,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":87425024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30656034,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":54,"num_read_kb":36263,"num_write":86,"num_write_kb":20088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039772"},"pg_stats":[{"pgid":"3.f","version":"412'129","reported_seq":420,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.283292+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:33.283292+0000","last_peered":"2023-03-29T14:09:33.283292+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:33.283292+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126894,"num_objects":28,"num_object_clones":28,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":65946,"num_write":172,"num_write_kb":51352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":454,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.347045+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:34.347045+0000","last_peered":"2023-03-29T14:09:34.347045+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:34.347045+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"425'262","reported_seq":952,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:33.273429+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:33.273429+0000","last_peered":"2023-03-29T14:09:33.273429+0000","last_clean":"2023-03-29T14:09:33.273429+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:33.273429+0000","last_undegraded":"2023-03-29T14:09:33.273429+0000","last_fullsized":"2023-03-29T14:09:33.273429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25951262,"num_objects":96,"num_object_clones":86,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":399,"num_read_kb":282257,"num_write":504,"num_write_kb":150092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"426'290","reported_seq":993,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:33.478203+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:33.478203+0000","last_peered":"2023-03-29T14:09:33.478203+0000","last_clean":"2023-03-29T14:09:33.478203+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:33.478203+0000","last_undegraded":"2023-03-29T14:09:33.478203+0000","last_fullsized":"2023-03-29T14:09:33.478203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22287534,"num_objects":116,"num_object_clones":108,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":394,"num_read_kb":231810,"num_write":570,"num_write_kb":160574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"427'224","reported_seq":838,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.411528+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:34.411528+0000","last_peered":"2023-03-29T14:09:34.411528+0000","last_clean":"2023-03-29T14:09:34.411528+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:34.411528+0000","last_undegraded":"2023-03-29T14:09:34.411528+0000","last_fullsized":"2023-03-29T14:09:34.411528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17345748,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":327,"num_read_kb":216072,"num_write":444,"num_write_kb":130902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":361,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.278883+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:33.278883+0000","last_peered":"2023-03-29T14:09:33.278883+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:33.278883+0000","last_undegraded":"2023-03-29T14:09:33.278883+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"427'160","reported_seq":690,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.344758+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:34.344758+0000","last_peered":"2023-03-29T14:09:34.344758+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:34.344758+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18992966,"num_objects":56,"num_object_clones":50,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":74217,"num_write":294,"num_write_kb":82448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":56}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":584,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.344693+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:34.344693+0000","last_peered":"2023-03-29T14:09:34.344693+0000","last_clean":"2023-03-29T14:09:34.344693+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:34.344693+0000","last_undegraded":"2023-03-29T14:09:34.344693+0000","last_fullsized":"2023-03-29T14:09:34.344693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":82092,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.370260+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:34.370260+0000","last_peered":"2023-03-29T14:09:34.370260+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:34.370260+0000","last_undegraded":"2023-03-29T14:09:34.370260+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"418'103","reported_seq":339,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.325171+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:33.325171+0000","last_peered":"2023-03-29T14:09:33.325171+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:33.325171+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12370612,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":34367,"num_write":130,"num_write_kb":35062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.276742+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:33.276742+0000","last_peered":"2023-03-29T14:09:33.276742+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:33.276742+0000","last_undegraded":"2023-03-29T14:09:33.276742+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"426'117","reported_seq":603,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.763162+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:33.763162+0000","last_peered":"2023-03-29T14:09:33.763162+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:33.763162+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12135380,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":167,"num_read_kb":107607,"num_write":230,"num_write_kb":68108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":194,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.381357+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:34.381357+0000","last_peered":"2023-03-29T14:09:34.381357+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:34.381357+0000","last_undegraded":"2023-03-29T14:09:34.381357+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":701,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.364381+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:34.364381+0000","last_peered":"2023-03-29T14:09:34.364381+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:34.364381+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.285195+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:33.285195+0000","last_peered":"2023-03-29T14:09:33.285195+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:33.285195+0000","last_undegraded":"2023-03-29T14:09:33.285195+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":558,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.345635+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:34.345635+0000","last_peered":"2023-03-29T14:09:34.345635+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:34.345635+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"424'216","reported_seq":882,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.277557+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:33.277557+0000","last_peered":"2023-03-29T14:09:33.277557+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:33.277557+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900544,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":376,"num_read_kb":253754,"num_write":420,"num_write_kb":121648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":80}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.284373+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:33.284373+0000","last_peered":"2023-03-29T14:09:33.284373+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:33.284373+0000","last_undegraded":"2023-03-29T14:09:33.284373+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"425'376","reported_seq":1185,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.386480+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:33.386480+0000","last_peered":"2023-03-29T14:09:33.386480+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:33.386480+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29380352,"num_objects":136,"num_object_clones":122,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":559,"num_read_kb":345140,"num_write":706,"num_write_kb":206376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":136}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.344282+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:34.344282+0000","last_peered":"2023-03-29T14:09:34.344282+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:34.344282+0000","last_undegraded":"2023-03-29T14:09:34.344282+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":591,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.353358+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:34.353358+0000","last_peered":"2023-03-29T14:09:34.353358+0000","last_clean":"2023-03-29T14:09:34.353358+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:34.353358+0000","last_undegraded":"2023-03-29T14:09:34.353358+0000","last_fullsized":"2023-03-29T14:09:34.353358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":88544,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"426'246","reported_seq":844,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.368552+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:34.368552+0000","last_peered":"2023-03-29T14:09:34.368552+0000","last_clean":"2023-03-29T14:09:34.368552+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:34.368552+0000","last_undegraded":"2023-03-29T14:09:34.368552+0000","last_fullsized":"2023-03-29T14:09:34.368552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278190,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":299,"num_read_kb":172367,"num_write":438,"num_write_kb":126954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.362370+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:34.362370+0000","last_peered":"2023-03-29T14:09:34.362370+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:34.362370+0000","last_undegraded":"2023-03-29T14:09:34.362370+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.279707+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:33.279707+0000","last_peered":"2023-03-29T14:09:33.279707+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:33.279707+0000","last_undegraded":"2023-03-29T14:09:33.279707+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"420'242","reported_seq":877,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.322307+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:33.322307+0000","last_peered":"2023-03-29T14:09:33.322307+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:33.322307+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22051774,"num_objects":88,"num_object_clones":80,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":341,"num_read_kb":213659,"num_write":460,"num_write_kb":137770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":88}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":240050840,"num_objects":1000,"num_object_clones":908,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3832,"num_read_kb":2353670,"num_write":5222,"num_write_kb":1521442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2810,"ondisk_log_size":2810,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":563676,"kb_used_data":539860,"kb_used_omap":0,"kb_used_meta":23808,"kb_avail":93808164,"statfs":{"total":96636764160,"available":96059559936,"internally_reserved":0,"allocated":552816640,"data_stored":689079797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24379392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706113,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":628068,"kb_used_data":602844,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":93743772,"statfs":{"total":96636764160,"available":95993622528,"internally_reserved":0,"allocated":617312256,"data_stored":770470945,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706113,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":569320,"kb_used_data":545056,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":93802520,"statfs":{"total":96636764160,"available":96053780480,"internally_reserved":0,"allocated":558137344,"data_stored":698804825,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:38.042 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-03-29T14:09:38.042 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:38.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:38.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:38.170+0000 7fcded465700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:38.177 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:38.176+0000 7fcded465700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:38.308 INFO:tasks.rados.rados.0.smithi017.stdout:1764: done (0 left) 2023-03-29T14:09:38.308 INFO:tasks.rados.rados.0.smithi017.stdout:1765: delete oid 18 current snap is 204 2023-03-29T14:09:38.357 INFO:tasks.rados.rados.0.smithi017.stdout:1765: done (0 left) 2023-03-29T14:09:38.358 INFO:tasks.rados.rados.0.smithi017.stdout:1766: snap_remove snap 174 2023-03-29T14:09:38.383 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:38.397 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":651,"stamp":"2023-03-29T14:09:38.306438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241231576,"num_objects":1004,"num_object_clones":908,"num_object_copies":3012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3974,"num_read_kb":2353790,"num_write":5496,"num_write_kb":1525230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2907,"ondisk_log_size":2907,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1917008,"kb_used_data":1831600,"kb_used_omap":0,"kb_used_meta":85376,"kb_avail":375570352,"statfs":{"total":386547056640,"available":384584040448,"internally_reserved":0,"allocated":1875558400,"data_stored":2331587792,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":87425024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30656034,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":54,"num_read_kb":36263,"num_write":86,"num_write_kb":20088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015551"},"pg_stats":[{"pgid":"3.f","version":"412'129","reported_seq":420,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.283292+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:33.283292+0000","last_peered":"2023-03-29T14:09:33.283292+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:33.283292+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126894,"num_objects":28,"num_object_clones":28,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":65946,"num_write":172,"num_write_kb":51352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":454,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.347045+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:34.347045+0000","last_peered":"2023-03-29T14:09:34.347045+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:34.347045+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"425'262","reported_seq":952,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:33.273429+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:33.273429+0000","last_peered":"2023-03-29T14:09:33.273429+0000","last_clean":"2023-03-29T14:09:33.273429+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:33.273429+0000","last_undegraded":"2023-03-29T14:09:33.273429+0000","last_fullsized":"2023-03-29T14:09:33.273429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25951262,"num_objects":96,"num_object_clones":86,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":399,"num_read_kb":282257,"num_write":504,"num_write_kb":150092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"426'290","reported_seq":993,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:33.478203+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:33.478203+0000","last_peered":"2023-03-29T14:09:33.478203+0000","last_clean":"2023-03-29T14:09:33.478203+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:33.478203+0000","last_undegraded":"2023-03-29T14:09:33.478203+0000","last_fullsized":"2023-03-29T14:09:33.478203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22287534,"num_objects":116,"num_object_clones":108,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":394,"num_read_kb":231810,"num_write":570,"num_write_kb":160574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"427'224","reported_seq":838,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.411528+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:34.411528+0000","last_peered":"2023-03-29T14:09:34.411528+0000","last_clean":"2023-03-29T14:09:34.411528+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:34.411528+0000","last_undegraded":"2023-03-29T14:09:34.411528+0000","last_fullsized":"2023-03-29T14:09:34.411528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17345748,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":327,"num_read_kb":216072,"num_write":444,"num_write_kb":130902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":361,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.278883+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:33.278883+0000","last_peered":"2023-03-29T14:09:33.278883+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:33.278883+0000","last_undegraded":"2023-03-29T14:09:33.278883+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"427'160","reported_seq":690,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.344758+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:34.344758+0000","last_peered":"2023-03-29T14:09:34.344758+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:34.344758+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18992966,"num_objects":56,"num_object_clones":50,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":74217,"num_write":294,"num_write_kb":82448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":56}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":584,"reported_epoch":427,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.344693+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:34.344693+0000","last_peered":"2023-03-29T14:09:34.344693+0000","last_clean":"2023-03-29T14:09:34.344693+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:34.344693+0000","last_undegraded":"2023-03-29T14:09:34.344693+0000","last_fullsized":"2023-03-29T14:09:34.344693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":82092,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.370260+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:34.370260+0000","last_peered":"2023-03-29T14:09:34.370260+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:34.370260+0000","last_undegraded":"2023-03-29T14:09:34.370260+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"418'103","reported_seq":339,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.325171+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:33.325171+0000","last_peered":"2023-03-29T14:09:33.325171+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:33.325171+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12370612,"num_objects":22,"num_object_clones":22,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":34367,"num_write":130,"num_write_kb":35062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":22}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.276742+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:33.276742+0000","last_peered":"2023-03-29T14:09:33.276742+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:33.276742+0000","last_undegraded":"2023-03-29T14:09:33.276742+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"426'117","reported_seq":603,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.763162+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:33.763162+0000","last_peered":"2023-03-29T14:09:33.763162+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:33.763162+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12135380,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":167,"num_read_kb":107607,"num_write":230,"num_write_kb":68108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":194,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.381357+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:34.381357+0000","last_peered":"2023-03-29T14:09:34.381357+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:34.381357+0000","last_undegraded":"2023-03-29T14:09:34.381357+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":701,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.364381+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:34.364381+0000","last_peered":"2023-03-29T14:09:34.364381+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:34.364381+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.285195+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:33.285195+0000","last_peered":"2023-03-29T14:09:33.285195+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:33.285195+0000","last_undegraded":"2023-03-29T14:09:33.285195+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":558,"reported_epoch":428,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:34.345635+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:34.345635+0000","last_peered":"2023-03-29T14:09:34.345635+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:34.345635+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"424'216","reported_seq":882,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.277557+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:33.277557+0000","last_peered":"2023-03-29T14:09:33.277557+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:33.277557+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900544,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":376,"num_read_kb":253754,"num_write":420,"num_write_kb":121648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":80}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.284373+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:33.284373+0000","last_peered":"2023-03-29T14:09:33.284373+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:33.284373+0000","last_undegraded":"2023-03-29T14:09:33.284373+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"425'376","reported_seq":1185,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.386480+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:33.386480+0000","last_peered":"2023-03-29T14:09:33.386480+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:33.386480+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29380352,"num_objects":136,"num_object_clones":122,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":559,"num_read_kb":345140,"num_write":706,"num_write_kb":206376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":136}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.344282+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:34.344282+0000","last_peered":"2023-03-29T14:09:34.344282+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:34.344282+0000","last_undegraded":"2023-03-29T14:09:34.344282+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":591,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.353358+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:34.353358+0000","last_peered":"2023-03-29T14:09:34.353358+0000","last_clean":"2023-03-29T14:09:34.353358+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:34.353358+0000","last_undegraded":"2023-03-29T14:09:34.353358+0000","last_fullsized":"2023-03-29T14:09:34.353358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":152,"num_read_kb":88544,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"426'246","reported_seq":844,"reported_epoch":428,"state":"active+clean","last_fresh":"2023-03-29T14:09:34.368552+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:34.368552+0000","last_peered":"2023-03-29T14:09:34.368552+0000","last_clean":"2023-03-29T14:09:34.368552+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:34.368552+0000","last_undegraded":"2023-03-29T14:09:34.368552+0000","last_fullsized":"2023-03-29T14:09:34.368552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278190,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":299,"num_read_kb":172367,"num_write":438,"num_write_kb":126954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":333,"reported_epoch":428,"state":"active+undersized","last_fresh":"2023-03-29T14:09:34.362370+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:34.362370+0000","last_peered":"2023-03-29T14:09:34.362370+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:34.362370+0000","last_undegraded":"2023-03-29T14:09:34.362370+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":364,"reported_epoch":427,"state":"active+undersized","last_fresh":"2023-03-29T14:09:33.279707+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:33.279707+0000","last_peered":"2023-03-29T14:09:33.279707+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:33.279707+0000","last_undegraded":"2023-03-29T14:09:33.279707+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"420'242","reported_seq":877,"reported_epoch":427,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:33.322307+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:33.322307+0000","last_peered":"2023-03-29T14:09:33.322307+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:33.322307+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22051774,"num_objects":88,"num_object_clones":80,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":341,"num_read_kb":213659,"num_write":460,"num_write_kb":137770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":88}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":240050840,"num_objects":1000,"num_object_clones":908,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":22,"num_read":3832,"num_read_kb":2353670,"num_write":5222,"num_write_kb":1521442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2810,"ondisk_log_size":2810,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706114,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":563676,"kb_used_data":539860,"kb_used_omap":0,"kb_used_meta":23808,"kb_avail":93808164,"statfs":{"total":96636764160,"available":96059559936,"internally_reserved":0,"allocated":552816640,"data_stored":689079797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24379392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706113,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":628068,"kb_used_data":602844,"kb_used_omap":0,"kb_used_meta":25216,"kb_avail":93743772,"statfs":{"total":96636764160,"available":95993622528,"internally_reserved":0,"allocated":617312256,"data_stored":770470945,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25821184},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706113,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":569320,"kb_used_data":545056,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":93802520,"statfs":{"total":96636764160,"available":96053780480,"internally_reserved":0,"allocated":558137344,"data_stored":698804825,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:39.311 INFO:tasks.rados.rados.0.smithi017.stdout:1766: done (0 left) 2023-03-29T14:09:39.311 INFO:tasks.rados.rados.0.smithi017.stdout:1767: write (excl) oid 30 current snap is 204 2023-03-29T14:09:39.312 INFO:tasks.rados.rados.0.smithi017.stdout:1767: seq_num 853 ranges {736371=418933,1798905=434297} 2023-03-29T14:09:39.317 INFO:tasks.rados.rados.0.smithi017.stdout:1767: writing smithi017107884-30 from 736371 to 1155304 tid 1 2023-03-29T14:09:39.324 INFO:tasks.rados.rados.0.smithi017.stdout:1767: writing smithi017107884-30 from 1798905 to 2233202 tid 2 2023-03-29T14:09:39.324 INFO:tasks.rados.rados.0.smithi017.stdout:1768: write (excl) oid 29 current snap is 204 2023-03-29T14:09:39.324 INFO:tasks.rados.rados.0.smithi017.stdout:1768: seq_num 854 ranges {753178=494182,1857387=495791} 2023-03-29T14:09:39.330 INFO:tasks.rados.rados.0.smithi017.stdout:1768: writing smithi017107884-29 from 753178 to 1247360 tid 1 2023-03-29T14:09:39.336 INFO:tasks.rados.rados.0.smithi017.stdout:1768: writing smithi017107884-29 from 1857387 to 2353178 tid 2 2023-03-29T14:09:39.337 INFO:tasks.rados.rados.0.smithi017.stdout:1769: read oid 45 snap -1 2023-03-29T14:09:39.337 INFO:tasks.rados.rados.0.smithi017.stdout:1769: expect deleted 2023-03-29T14:09:39.337 INFO:tasks.rados.rados.0.smithi017.stdout:1770: delete oid 37 current snap is 204 2023-03-29T14:09:39.449 INFO:tasks.rados.rados.0.smithi017.stdout:1767: finishing write tid 1 to smithi017107884-30 2023-03-29T14:09:39.467 INFO:tasks.rados.rados.0.smithi017.stdout:1767: finishing write tid 2 to smithi017107884-30 2023-03-29T14:09:39.470 INFO:tasks.rados.rados.0.smithi017.stdout:1767: finishing write tid 3 to smithi017107884-30 2023-03-29T14:09:39.472 INFO:tasks.rados.rados.0.smithi017.stdout:1768: finishing write tid 1 to smithi017107884-29 2023-03-29T14:09:39.488 INFO:tasks.rados.rados.0.smithi017.stdout:1767: finishing write tid 4 to smithi017107884-30 2023-03-29T14:09:39.489 INFO:tasks.rados.rados.0.smithi017.stdout:1767: oid 30 updating version 0 to 107 2023-03-29T14:09:39.489 INFO:tasks.rados.rados.0.smithi017.stdout:1767: oid 30 version 107 is already newer than 107 2023-03-29T14:09:39.489 INFO:tasks.rados.rados.0.smithi017.stdout:1767: oid 30 version 107 is already newer than 104 2023-03-29T14:09:39.489 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 30 v 107 (ObjNum 853 snap 204 seq_num 853) dirty exists 2023-03-29T14:09:39.489 INFO:tasks.rados.rados.0.smithi017.stdout:1767: left oid 30 (ObjNum 853 snap 204 seq_num 853) 2023-03-29T14:09:39.508 INFO:tasks.rados.rados.0.smithi017.stdout:1768: finishing write tid 2 to smithi017107884-29 2023-03-29T14:09:39.535 INFO:tasks.rados.rados.0.smithi017.stdout:1767: done (3 left) 2023-03-29T14:09:39.535 INFO:tasks.rados.rados.0.smithi017.stdout:1769: done (2 left) 2023-03-29T14:09:39.535 INFO:tasks.rados.rados.0.smithi017.stdout:1770: done (1 left) 2023-03-29T14:09:39.535 INFO:tasks.rados.rados.0.smithi017.stdout:1771: write (excl) oid 22 current snap is 204 2023-03-29T14:09:39.535 INFO:tasks.rados.rados.0.smithi017.stdout:1771: seq_num 856 ranges {786792=644680,1857998=528794} 2023-03-29T14:09:39.543 INFO:tasks.rados.rados.0.smithi017.stdout:1771: writing smithi017107884-22 from 786792 to 1431472 tid 1 2023-03-29T14:09:39.548 INFO:tasks.rados.rados.0.smithi017.stdout:1771: writing smithi017107884-22 from 1857998 to 2386792 tid 2 2023-03-29T14:09:39.548 INFO:tasks.rados.rados.0.smithi017.stdout:1772: read oid 17 snap 197 2023-03-29T14:09:39.548 INFO:tasks.rados.rados.0.smithi017.stdout:1772: expect (ObjNum 764 snap 188 seq_num 764) 2023-03-29T14:09:39.548 INFO:tasks.rados.rados.0.smithi017.stdout:1773: read oid 44 snap -1 2023-03-29T14:09:39.549 INFO:tasks.rados.rados.0.smithi017.stdout:1773: expect deleted 2023-03-29T14:09:39.549 INFO:tasks.rados.rados.0.smithi017.stdout:1774: snap_create 2023-03-29T14:09:39.549 INFO:tasks.rados.rados.0.smithi017.stdout:1768: finishing write tid 3 to smithi017107884-29 2023-03-29T14:09:39.551 INFO:tasks.rados.rados.0.smithi017.stdout:1768: finishing write tid 4 to smithi017107884-29 2023-03-29T14:09:39.551 INFO:tasks.rados.rados.0.smithi017.stdout:1768: oid 29 updating version 0 to 118 2023-03-29T14:09:39.551 INFO:tasks.rados.rados.0.smithi017.stdout:1768: oid 29 updating version 118 to 120 2023-03-29T14:09:39.551 INFO:tasks.rados.rados.0.smithi017.stdout:1768: oid 29 version 120 is already newer than 120 2023-03-29T14:09:39.551 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 120 (ObjNum 854 snap 204 seq_num 854) dirty exists 2023-03-29T14:09:39.551 INFO:tasks.rados.rados.0.smithi017.stdout:1768: left oid 29 (ObjNum 854 snap 204 seq_num 854) 2023-03-29T14:09:39.601 INFO:tasks.rados.rados.0.smithi017.stdout:1772: expect (ObjNum 764 snap 188 seq_num 764) 2023-03-29T14:09:39.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~270s 2023-03-29T14:09:39.790 INFO:tasks.rados.rados.0.smithi017.stdout:1771: finishing write tid 1 to smithi017107884-22 2023-03-29T14:09:39.790 INFO:tasks.rados.rados.0.smithi017.stdout:1771: finishing write tid 2 to smithi017107884-22 2023-03-29T14:09:39.790 INFO:tasks.rados.rados.0.smithi017.stdout:1771: finishing write tid 3 to smithi017107884-22 2023-03-29T14:09:39.791 INFO:tasks.rados.rados.0.smithi017.stdout:1771: finishing write tid 4 to smithi017107884-22 2023-03-29T14:09:39.791 INFO:tasks.rados.rados.0.smithi017.stdout:1771: oid 22 updating version 0 to 130 2023-03-29T14:09:39.791 INFO:tasks.rados.rados.0.smithi017.stdout:1771: oid 22 updating version 130 to 133 2023-03-29T14:09:39.791 INFO:tasks.rados.rados.0.smithi017.stdout:1771: oid 22 version 133 is already newer than 133 2023-03-29T14:09:39.791 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 22 v 133 (ObjNum 856 snap 204 seq_num 856) dirty exists 2023-03-29T14:09:39.791 INFO:tasks.rados.rados.0.smithi017.stdout:1771: left oid 22 (ObjNum 856 snap 204 seq_num 856) 2023-03-29T14:09:39.927 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:40.058 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:40.056+0000 7f332ce46700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:40.061 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:40.060+0000 7f332ce46700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:40.124 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:40.127 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:40.127 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:40.315 INFO:tasks.rados.rados.0.smithi017.stdout:1768: done (4 left) 2023-03-29T14:09:40.315 INFO:tasks.rados.rados.0.smithi017.stdout:1771: done (3 left) 2023-03-29T14:09:40.315 INFO:tasks.rados.rados.0.smithi017.stdout:1772: done (2 left) 2023-03-29T14:09:40.315 INFO:tasks.rados.rados.0.smithi017.stdout:1773: done (1 left) 2023-03-29T14:09:40.315 INFO:tasks.rados.rados.0.smithi017.stdout:1774: done (0 left) 2023-03-29T14:09:40.316 INFO:tasks.rados.rados.0.smithi017.stdout:1775: read oid 35 snap -1 2023-03-29T14:09:40.316 INFO:tasks.rados.rados.0.smithi017.stdout:1775: expect (ObjNum 786 snap 191 seq_num 786) 2023-03-29T14:09:40.316 INFO:tasks.rados.rados.0.smithi017.stdout:1776: read oid 5 snap -1 2023-03-29T14:09:40.316 INFO:tasks.rados.rados.0.smithi017.stdout:1776: expect deleted 2023-03-29T14:09:40.316 INFO:tasks.rados.rados.0.smithi017.stdout:1777: delete oid 11 current snap is 205 2023-03-29T14:09:40.400 INFO:tasks.rados.rados.0.smithi017.stdout:1777: done (2 left) 2023-03-29T14:09:40.400 INFO:tasks.rados.rados.0.smithi017.stdout:1778: delete oid 31 current snap is 205 2023-03-29T14:09:40.433 INFO:tasks.rados.rados.0.smithi017.stdout:1775: expect (ObjNum 786 snap 191 seq_num 786) 2023-03-29T14:09:40.768 INFO:tasks.rados.rados.0.smithi017.stdout:1775: done (2 left) 2023-03-29T14:09:40.768 INFO:tasks.rados.rados.0.smithi017.stdout:1776: done (1 left) 2023-03-29T14:09:40.768 INFO:tasks.rados.rados.0.smithi017.stdout:1778: done (0 left) 2023-03-29T14:09:40.769 INFO:tasks.rados.rados.0.smithi017.stdout:1779: write (excl) oid 20 current snap is 205 2023-03-29T14:09:40.769 INFO:tasks.rados.rados.0.smithi017.stdout:1779: seq_num 859 ranges {437213=470427,1633444=728951} 2023-03-29T14:09:40.773 INFO:tasks.rados.rados.0.smithi017.stdout:1779: writing smithi017107884-20 from 437213 to 907640 tid 1 2023-03-29T14:09:40.781 INFO:tasks.rados.rados.0.smithi017.stdout:1779: writing smithi017107884-20 from 1633444 to 2362395 tid 2 2023-03-29T14:09:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:1780: read oid 26 snap -1 2023-03-29T14:09:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:1780: expect (ObjNum 825 snap 199 seq_num 825) 2023-03-29T14:09:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:1781: read oid 34 snap -1 2023-03-29T14:09:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:1781: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:09:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:1782: read oid 21 snap -1 2023-03-29T14:09:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:1782: expect (ObjNum 837 snap 202 seq_num 837) 2023-03-29T14:09:40.782 INFO:tasks.rados.rados.0.smithi017.stdout:1783: snap_remove snap 204 2023-03-29T14:09:41.318 INFO:tasks.rados.rados.0.smithi017.stdout:1783: done (4 left) 2023-03-29T14:09:41.319 INFO:tasks.rados.rados.0.smithi017.stdout:1784: write (excl) oid 37 current snap is 205 2023-03-29T14:09:41.319 INFO:tasks.rados.rados.0.smithi017.stdout:1784: seq_num 860 ranges {454020=662029,1691926=588962} 2023-03-29T14:09:41.324 INFO:tasks.rados.rados.0.smithi017.stdout:1784: writing smithi017107884-37 from 454020 to 1116049 tid 1 2023-03-29T14:09:41.331 INFO:tasks.rados.rados.0.smithi017.stdout:1784: writing smithi017107884-37 from 1691926 to 2280888 tid 2 2023-03-29T14:09:41.331 INFO:tasks.rados.rados.0.smithi017.stdout:1785: write (excl) oid 44 current snap is 205 2023-03-29T14:09:41.332 INFO:tasks.rados.rados.0.smithi017.stdout:1785: seq_num 861 ranges {470827=737278,1750408=720419} 2023-03-29T14:09:41.338 INFO:tasks.rados.rados.0.smithi017.stdout:1785: writing smithi017107884-44 from 470827 to 1208105 tid 1 2023-03-29T14:09:41.345 INFO:tasks.rados.rados.0.smithi017.stdout:1785: writing smithi017107884-44 from 1750408 to 2470827 tid 2 2023-03-29T14:09:41.345 INFO:tasks.rados.rados.0.smithi017.stdout:1786: delete oid 18 current snap is 205 2023-03-29T14:09:41.345 INFO:tasks.rados.rados.0.smithi017.stdout:1780: expect (ObjNum 825 snap 199 seq_num 825) 2023-03-29T14:09:41.555 INFO:tasks.rados.rados.0.smithi017.stdout:1779: finishing write tid 1 to smithi017107884-20 2023-03-29T14:09:41.556 INFO:tasks.rados.rados.0.smithi017.stdout:1780: done (6 left) 2023-03-29T14:09:41.556 INFO:tasks.rados.rados.0.smithi017.stdout:1786: done (5 left) 2023-03-29T14:09:41.556 INFO:tasks.rados.rados.0.smithi017.stdout:1787: delete oid 2 current snap is 205 2023-03-29T14:09:41.556 INFO:tasks.rados.rados.0.smithi017.stdout:1782: expect (ObjNum 837 snap 202 seq_num 837) 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: finishing write tid 2 to smithi017107884-20 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: finishing write tid 3 to smithi017107884-20 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: finishing write tid 4 to smithi017107884-20 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: oid 20 updating version 0 to 228 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: oid 20 version 228 is already newer than 228 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: oid 20 version 228 is already newer than 225 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 228 (ObjNum 859 snap 205 seq_num 859) dirty exists 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: left oid 20 (ObjNum 859 snap 205 seq_num 859) 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1779: done (5 left) 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1782: done (4 left) 2023-03-29T14:09:41.657 INFO:tasks.rados.rados.0.smithi017.stdout:1787: done (3 left) 2023-03-29T14:09:41.658 INFO:tasks.rados.rados.0.smithi017.stdout:1788: read oid 18 snap -1 2023-03-29T14:09:41.658 INFO:tasks.rados.rados.0.smithi017.stdout:1788: expect deleted 2023-03-29T14:09:41.658 INFO:tasks.rados.rados.0.smithi017.stdout:1789: read oid 2 snap -1 2023-03-29T14:09:41.658 INFO:tasks.rados.rados.0.smithi017.stdout:1789: expect deleted 2023-03-29T14:09:41.658 INFO:tasks.rados.rados.0.smithi017.stdout:1790: write oid 39 current snap is 205 2023-03-29T14:09:41.658 INFO:tasks.rados.rados.0.smithi017.stdout:1781: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:09:41.742 INFO:tasks.rados.rados.0.smithi017.stdout:1784: finishing write tid 1 to smithi017107884-37 2023-03-29T14:09:41.742 INFO:tasks.rados.rados.0.smithi017.stdout:1784: finishing write tid 2 to smithi017107884-37 2023-03-29T14:09:41.742 INFO:tasks.rados.rados.0.smithi017.stdout:1785: finishing write tid 1 to smithi017107884-44 2023-03-29T14:09:41.742 INFO:tasks.rados.rados.0.smithi017.stdout:1785: finishing write tid 2 to smithi017107884-44 2023-03-29T14:09:41.742 INFO:tasks.rados.rados.0.smithi017.stdout:1784: finishing write tid 3 to smithi017107884-37 2023-03-29T14:09:41.742 INFO:tasks.rados.rados.0.smithi017.stdout:1790: seq_num 864 ranges {521248=563025,1809501=596300} 2023-03-29T14:09:41.748 INFO:tasks.rados.rados.0.smithi017.stdout:1790: writing smithi017107884-39 from 521248 to 1084273 tid 1 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1790: writing smithi017107884-39 from 1809501 to 2405801 tid 2 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1781: done (5 left) 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1791: write (excl) oid 20 current snap is 205 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1784: finishing write tid 4 to smithi017107884-37 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1784: oid 37 updating version 0 to 109 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1784: oid 37 updating version 109 to 111 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1784: oid 37 version 111 is already newer than 111 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 37 v 111 (ObjNum 860 snap 205 seq_num 860) dirty exists 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1784: left oid 37 (ObjNum 860 snap 205 seq_num 860) 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1785: finishing write tid 3 to smithi017107884-44 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1785: finishing write tid 4 to smithi017107884-44 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1785: oid 44 updating version 0 to 122 2023-03-29T14:09:41.756 INFO:tasks.rados.rados.0.smithi017.stdout:1785: oid 44 updating version 122 to 124 2023-03-29T14:09:41.757 INFO:tasks.rados.rados.0.smithi017.stdout:1785: oid 44 version 124 is already newer than 124 2023-03-29T14:09:41.757 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 44 v 124 (ObjNum 861 snap 205 seq_num 861) dirty exists 2023-03-29T14:09:41.757 INFO:tasks.rados.rados.0.smithi017.stdout:1785: left oid 44 (ObjNum 861 snap 205 seq_num 861) 2023-03-29T14:09:41.757 INFO:tasks.rados.rados.0.smithi017.stdout:1791: seq_num 865 ranges {538055=638274,1867983=670072} 2023-03-29T14:09:41.762 INFO:tasks.rados.rados.0.smithi017.stdout:1791: writing smithi017107884-20 from 538055 to 1176329 tid 1 2023-03-29T14:09:41.770 INFO:tasks.rados.rados.0.smithi017.stdout:1791: writing smithi017107884-20 from 1867983 to 2538055 tid 2 2023-03-29T14:09:41.770 INFO:tasks.rados.rados.0.smithi017.stdout:1784: done (5 left) 2023-03-29T14:09:41.770 INFO:tasks.rados.rados.0.smithi017.stdout:1785: done (4 left) 2023-03-29T14:09:41.770 INFO:tasks.rados.rados.0.smithi017.stdout:1788: done (3 left) 2023-03-29T14:09:41.770 INFO:tasks.rados.rados.0.smithi017.stdout:1789: done (2 left) 2023-03-29T14:09:41.770 INFO:tasks.rados.rados.0.smithi017.stdout:1792: write oid 14 current snap is 205 2023-03-29T14:09:41.770 INFO:tasks.rados.rados.0.smithi017.stdout:1792: seq_num 866 ranges {554862=713523,1926465=599969} 2023-03-29T14:09:41.777 INFO:tasks.rados.rados.0.smithi017.stdout:1792: writing smithi017107884-14 from 554862 to 1268385 tid 1 2023-03-29T14:09:41.784 INFO:tasks.rados.rados.0.smithi017.stdout:1792: writing smithi017107884-14 from 1926465 to 2526434 tid 2 2023-03-29T14:09:41.784 INFO:tasks.rados.rados.0.smithi017.stdout:1793: write oid 29 current snap is 205 2023-03-29T14:09:41.784 INFO:tasks.rados.rados.0.smithi017.stdout:1790: finishing write tid 1 to smithi017107884-39 2023-03-29T14:09:41.784 INFO:tasks.rados.rados.0.smithi017.stdout:1793: seq_num 867 ranges {571669=505125,1701300=743627} 2023-03-29T14:09:41.790 INFO:tasks.rados.rados.0.smithi017.stdout:1793: writing smithi017107884-29 from 571669 to 1076794 tid 1 2023-03-29T14:09:41.798 INFO:tasks.rados.rados.0.smithi017.stdout:1793: writing smithi017107884-29 from 1701300 to 2444927 tid 2 2023-03-29T14:09:41.798 INFO:tasks.rados.rados.0.smithi017.stdout:1794: read oid 23 snap 197 2023-03-29T14:09:41.798 INFO:tasks.rados.rados.0.smithi017.stdout:1794: expect (ObjNum 797 snap 194 seq_num 797) 2023-03-29T14:09:41.798 INFO:tasks.rados.rados.0.smithi017.stdout:1795: read oid 46 snap 197 2023-03-29T14:09:41.799 INFO:tasks.rados.rados.0.smithi017.stdout:1795: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:41.799 INFO:tasks.rados.rados.0.smithi017.stdout:1796: snap_create 2023-03-29T14:09:41.820 INFO:tasks.rados.rados.0.smithi017.stdout:1790: finishing write tid 2 to smithi017107884-39 2023-03-29T14:09:41.861 INFO:tasks.rados.rados.0.smithi017.stdout:1790: finishing write tid 3 to smithi017107884-39 2023-03-29T14:09:41.861 INFO:tasks.rados.rados.0.smithi017.stdout:1790: finishing write tid 4 to smithi017107884-39 2023-03-29T14:09:41.861 INFO:tasks.rados.rados.0.smithi017.stdout:1790: oid 39 updating version 0 to 266 2023-03-29T14:09:41.861 INFO:tasks.rados.rados.0.smithi017.stdout:1790: oid 39 version 266 is already newer than 266 2023-03-29T14:09:41.861 INFO:tasks.rados.rados.0.smithi017.stdout:1790: oid 39 version 266 is already newer than 266 2023-03-29T14:09:41.861 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 39 v 266 (ObjNum 864 snap 205 seq_num 864) dirty exists 2023-03-29T14:09:41.861 INFO:tasks.rados.rados.0.smithi017.stdout:1790: left oid 39 (ObjNum 864 snap 205 seq_num 864) 2023-03-29T14:09:41.867 INFO:tasks.rados.rados.0.smithi017.stdout:1792: finishing write tid 1 to smithi017107884-14 2023-03-29T14:09:41.873 INFO:tasks.rados.rados.0.smithi017.stdout:1793: finishing write tid 1 to smithi017107884-29 2023-03-29T14:09:41.879 INFO:tasks.rados.rados.0.smithi017.stdout:1792: finishing write tid 2 to smithi017107884-14 2023-03-29T14:09:41.888 INFO:tasks.rados.rados.0.smithi017.stdout:1792: finishing write tid 3 to smithi017107884-14 2023-03-29T14:09:41.889 INFO:tasks.rados.rados.0.smithi017.stdout:1794: expect (ObjNum 797 snap 194 seq_num 797) 2023-03-29T14:09:42.078 INFO:tasks.rados.rados.0.smithi017.stdout:1792: finishing write tid 4 to smithi017107884-14 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:1792: oid 14 updating version 0 to 168 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:1792: oid 14 version 168 is already newer than 168 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:1792: oid 14 version 168 is already newer than 168 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 168 (ObjNum 866 snap 205 seq_num 866) dirty exists 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:1792: left oid 14 (ObjNum 866 snap 205 seq_num 866) 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:1791: finishing write tid 1 to smithi017107884-20 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:1793: finishing write tid 2 to smithi017107884-29 2023-03-29T14:09:42.079 INFO:tasks.rados.rados.0.smithi017.stdout:1793: finishing write tid 3 to smithi017107884-29 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:1793: finishing write tid 4 to smithi017107884-29 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:1793: oid 29 updating version 0 to 123 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:1793: oid 29 version 123 is already newer than 123 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:1793: oid 29 version 123 is already newer than 123 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 29 v 123 (ObjNum 867 snap 205 seq_num 867) dirty exists 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:1793: left oid 29 (ObjNum 867 snap 205 seq_num 867) 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:1791: finishing write tid 2 to smithi017107884-20 2023-03-29T14:09:42.080 INFO:tasks.rados.rados.0.smithi017.stdout:1791: finishing write tid 3 to smithi017107884-20 2023-03-29T14:09:42.081 INFO:tasks.rados.rados.0.smithi017.stdout:1791: finishing write tid 4 to smithi017107884-20 2023-03-29T14:09:42.081 INFO:tasks.rados.rados.0.smithi017.stdout:1791: oid 20 updating version 0 to 231 2023-03-29T14:09:42.081 INFO:tasks.rados.rados.0.smithi017.stdout:1791: oid 20 version 231 is already newer than 229 2023-03-29T14:09:42.081 INFO:tasks.rados.rados.0.smithi017.stdout:1791: oid 20 version 231 is already newer than 231 2023-03-29T14:09:42.081 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 20 v 231 (ObjNum 865 snap 205 seq_num 865) dirty exists 2023-03-29T14:09:42.081 INFO:tasks.rados.rados.0.smithi017.stdout:1791: left oid 20 (ObjNum 865 snap 205 seq_num 865) 2023-03-29T14:09:42.081 INFO:tasks.rados.rados.0.smithi017.stdout:1795: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:42.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:42.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:42.257+0000 7f4adbfea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:42.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:42.261+0000 7f4adbfea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:42.322 INFO:tasks.rados.rados.0.smithi017.stdout:1790: done (6 left) 2023-03-29T14:09:42.323 INFO:tasks.rados.rados.0.smithi017.stdout:1791: done (5 left) 2023-03-29T14:09:42.323 INFO:tasks.rados.rados.0.smithi017.stdout:1792: done (4 left) 2023-03-29T14:09:42.323 INFO:tasks.rados.rados.0.smithi017.stdout:1793: done (3 left) 2023-03-29T14:09:42.323 INFO:tasks.rados.rados.0.smithi017.stdout:1794: done (2 left) 2023-03-29T14:09:42.323 INFO:tasks.rados.rados.0.smithi017.stdout:1795: done (1 left) 2023-03-29T14:09:42.323 INFO:tasks.rados.rados.0.smithi017.stdout:1796: done (0 left) 2023-03-29T14:09:42.323 INFO:tasks.rados.rados.0.smithi017.stdout:1797: snap_remove snap 197 2023-03-29T14:09:42.356 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:42.359 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:42.360 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:43.265 INFO:tasks.rados.rados.0.smithi017.stdout:1797: done (0 left) 2023-03-29T14:09:43.265 INFO:tasks.rados.rados.0.smithi017.stdout:1798: read oid 30 snap 205 2023-03-29T14:09:43.265 INFO:tasks.rados.rados.0.smithi017.stdout:1798: expect (ObjNum 853 snap 204 seq_num 853) 2023-03-29T14:09:43.265 INFO:tasks.rados.rados.0.smithi017.stdout:1799: read oid 40 snap -1 2023-03-29T14:09:43.265 INFO:tasks.rados.rados.0.smithi017.stdout:1799: expect (ObjNum 766 snap 188 seq_num 766) 2023-03-29T14:09:43.265 INFO:tasks.rados.rados.0.smithi017.stdout:1800: snap_create 2023-03-29T14:09:43.307 INFO:tasks.rados.rados.0.smithi017.stdout:1799: expect (ObjNum 766 snap 188 seq_num 766) 2023-03-29T14:09:43.401 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:43.402 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:43.402 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:43.516 INFO:tasks.rados.rados.0.smithi017.stdout:1798: expect (ObjNum 853 snap 204 seq_num 853) 2023-03-29T14:09:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:43.536+0000 7f61ab229700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:43.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:43.543+0000 7f61ab229700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:43.749 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:43.751 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:43.762 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":658,"stamp":"2023-03-29T14:09:43.262733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":195447988,"num_objects":1016,"num_object_clones":920,"num_object_copies":3048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":4035,"num_read_kb":2391066,"num_write":5528,"num_write_kb":1533474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2928,"ondisk_log_size":2928,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1916072,"kb_used_data":1827880,"kb_used_omap":0,"kb_used_meta":85696,"kb_avail":375571288,"statfs":{"total":386547056640,"available":384584998912,"internally_reserved":0,"allocated":1871749120,"data_stored":2327538626,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":87752704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-45783588,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":10,"num_read":61,"num_read_kb":37276,"num_write":32,"num_write_kb":8244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.952237"},"pg_stats":[{"pgid":"3.f","version":"432'133","reported_seq":432,"reported_epoch":434,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:40.417739+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:40.417739+0000","last_peered":"2023-03-29T14:09:40.417739+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:40.417739+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202608,"num_objects":30,"num_object_clones":30,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65947,"num_write":180,"num_write_kb":53646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":458,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:39.379279+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:39.379279+0000","last_peered":"2023-03-29T14:09:39.379279+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:39.379279+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"425'262","reported_seq":960,"reported_epoch":434,"state":"active+clean","last_fresh":"2023-03-29T14:09:40.410626+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:40.410626+0000","last_peered":"2023-03-29T14:09:40.410626+0000","last_clean":"2023-03-29T14:09:40.410626+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:40.410626+0000","last_undegraded":"2023-03-29T14:09:40.410626+0000","last_fullsized":"2023-03-29T14:09:40.410626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25951262,"num_objects":96,"num_object_clones":86,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":406,"num_read_kb":289764,"num_write":504,"num_write_kb":150092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"430'291","reported_seq":1000,"reported_epoch":434,"state":"active+clean","last_fresh":"2023-03-29T14:09:40.420902+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:40.420902+0000","last_peered":"2023-03-29T14:09:40.420902+0000","last_clean":"2023-03-29T14:09:40.420902+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:40.420902+0000","last_undegraded":"2023-03-29T14:09:40.420902+0000","last_fullsized":"2023-03-29T14:09:40.420902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14320686,"num_objects":116,"num_object_clones":108,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":396,"num_read_kb":231810,"num_write":570,"num_write_kb":160574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"433'228","reported_seq":852,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-03-29T14:09:40.896217+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:40.896217+0000","last_peered":"2023-03-29T14:09:40.896217+0000","last_clean":"2023-03-29T14:09:40.896217+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:40.896217+0000","last_undegraded":"2023-03-29T14:09:40.896217+0000","last_fullsized":"2023-03-29T14:09:40.896217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15194518,"num_objects":82,"num_object_clones":74,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":337,"num_read_kb":220426,"num_write":452,"num_write_kb":133246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":366,"reported_epoch":434,"state":"active+undersized","last_fresh":"2023-03-29T14:09:40.415944+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:40.415944+0000","last_peered":"2023-03-29T14:09:40.415944+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:40.415944+0000","last_undegraded":"2023-03-29T14:09:40.415944+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"433'164","reported_seq":700,"reported_epoch":434,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:41.341052+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:41.341052+0000","last_peered":"2023-03-29T14:09:41.341052+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:41.341052+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4538286,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":174,"num_read_kb":74217,"num_write":294,"num_write_kb":82448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":60}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":589,"reported_epoch":434,"state":"active+clean","last_fresh":"2023-03-29T14:09:40.424793+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:40.424793+0000","last_peered":"2023-03-29T14:09:40.424793+0000","last_clean":"2023-03-29T14:09:40.424793+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:40.424793+0000","last_undegraded":"2023-03-29T14:09:40.424793+0000","last_fullsized":"2023-03-29T14:09:40.424793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":82092,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":337,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-03-29T14:09:39.380821+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:39.380821+0000","last_peered":"2023-03-29T14:09:39.380821+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:39.380821+0000","last_undegraded":"2023-03-29T14:09:39.380821+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"432'108","reported_seq":353,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:39.484305+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:39.484305+0000","last_peered":"2023-03-29T14:09:39.484305+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:39.484305+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4672742,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":71,"num_read_kb":34368,"num_write":138,"num_write_kb":36732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":369,"reported_epoch":434,"state":"active+undersized","last_fresh":"2023-03-29T14:09:40.422144+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:40.422144+0000","last_peered":"2023-03-29T14:09:40.422144+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:40.422144+0000","last_undegraded":"2023-03-29T14:09:40.422144+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"432'120","reported_seq":615,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:39.550430+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:39.550430+0000","last_peered":"2023-03-29T14:09:39.550430+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:39.550430+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9177414,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107608,"num_write":238,"num_write_kb":70044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":198,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-03-29T14:09:39.354597+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:39.354597+0000","last_peered":"2023-03-29T14:09:39.354597+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:39.354597+0000","last_undegraded":"2023-03-29T14:09:39.354597+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":705,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:39.358156+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:39.358156+0000","last_peered":"2023-03-29T14:09:39.358156+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:39.358156+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":369,"reported_epoch":434,"state":"active+undersized","last_fresh":"2023-03-29T14:09:40.409832+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:40.409832+0000","last_peered":"2023-03-29T14:09:40.409832+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:40.409832+0000","last_undegraded":"2023-03-29T14:09:40.409832+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":562,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:39.388918+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:39.388918+0000","last_peered":"2023-03-29T14:09:39.388918+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:39.388918+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"424'216","reported_seq":893,"reported_epoch":434,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:40.820363+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:40.820363+0000","last_peered":"2023-03-29T14:09:40.820363+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:40.820363+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900544,"num_objects":80,"num_object_clones":72,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":390,"num_read_kb":264972,"num_write":420,"num_write_kb":121648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":80}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":369,"reported_epoch":434,"state":"active+undersized","last_fresh":"2023-03-29T14:09:40.419652+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:40.419652+0000","last_peered":"2023-03-29T14:09:40.419652+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:40.419652+0000","last_undegraded":"2023-03-29T14:09:40.419652+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"425'376","reported_seq":1192,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:40.379467+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:40.379467+0000","last_peered":"2023-03-29T14:09:40.379467+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:40.379467+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749644,"num_objects":138,"num_object_clones":124,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":559,"num_read_kb":345140,"num_write":706,"num_write_kb":206376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":337,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-03-29T14:09:39.360618+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:39.360618+0000","last_peered":"2023-03-29T14:09:39.360618+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:39.360618+0000","last_undegraded":"2023-03-29T14:09:39.360618+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"402'121","reported_seq":598,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-03-29T14:09:40.421646+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:40.421646+0000","last_peered":"2023-03-29T14:09:40.421646+0000","last_clean":"2023-03-29T14:09:40.421646+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:40.421646+0000","last_undegraded":"2023-03-29T14:09:40.421646+0000","last_fullsized":"2023-03-29T14:09:40.421646+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6420604,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":159,"num_read_kb":97290,"num_write":244,"num_write_kb":70776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"426'246","reported_seq":851,"reported_epoch":433,"state":"active+clean","last_fresh":"2023-03-29T14:09:39.372437+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:39.372437+0000","last_peered":"2023-03-29T14:09:39.372437+0000","last_clean":"2023-03-29T14:09:39.372437+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:39.372437+0000","last_undegraded":"2023-03-29T14:09:39.372437+0000","last_fullsized":"2023-03-29T14:09:39.372437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278190,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":306,"num_read_kb":177815,"num_write":438,"num_write_kb":126954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":337,"reported_epoch":433,"state":"active+undersized","last_fresh":"2023-03-29T14:09:39.350020+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:39.350020+0000","last_peered":"2023-03-29T14:09:39.350020+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:39.350020+0000","last_undegraded":"2023-03-29T14:09:39.350020+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":369,"reported_epoch":434,"state":"active+undersized","last_fresh":"2023-03-29T14:09:40.414648+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:40.414648+0000","last_peered":"2023-03-29T14:09:40.414648+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:40.414648+0000","last_undegraded":"2023-03-29T14:09:40.414648+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"420'242","reported_seq":882,"reported_epoch":433,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:39.382064+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:39.382064+0000","last_peered":"2023-03-29T14:09:39.382064+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:39.382064+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22051774,"num_objects":88,"num_object_clones":80,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":341,"num_read_kb":213659,"num_write":460,"num_write_kb":137770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":88}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":194267252,"num_objects":1012,"num_object_clones":920,"num_object_copies":3036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":550,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":3893,"num_read_kb":2390946,"num_write":5254,"num_write_kb":1529686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2831,"ondisk_log_size":2831,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706119,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":563616,"kb_used_data":539672,"kb_used_omap":0,"kb_used_meta":23936,"kb_avail":93808224,"statfs":{"total":96636764160,"available":96059621376,"internally_reserved":0,"allocated":552624128,"data_stored":689804154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24510464},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706118,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":627788,"kb_used_data":601204,"kb_used_omap":0,"kb_used_meta":25344,"kb_avail":93744052,"statfs":{"total":96636764160,"available":95993909248,"internally_reserved":0,"allocated":615632896,"data_stored":768909258,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25952256},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706118,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":568724,"kb_used_data":543164,"kb_used_omap":0,"kb_used_meta":24320,"kb_avail":93803116,"statfs":{"total":96636764160,"available":96054390784,"internally_reserved":0,"allocated":556199936,"data_stored":695592989,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":24903680},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:43.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.3 3.0 1.0 3.6 2023-03-29T14:09:43.904 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:43.895+0000 7ff879209700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:43.908 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:43.907+0000 7ff879209700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.3,3.6] on osd.0 to force-recovery; instructing pg(s) [1.0,3.0] on osd.1 to force-recovery; instructing pg(s) [3.f] on osd.3 to force-recovery; 2023-03-29T14:09:44.114 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:09:44.268 INFO:tasks.rados.rados.0.smithi017.stdout:1798: done (2 left) 2023-03-29T14:09:44.269 INFO:tasks.rados.rados.0.smithi017.stdout:1799: done (1 left) 2023-03-29T14:09:44.269 INFO:tasks.rados.rados.0.smithi017.stdout:1800: done (0 left) 2023-03-29T14:09:44.269 INFO:tasks.rados.rados.0.smithi017.stdout:1801: write oid 9 current snap is 207 2023-03-29T14:09:44.269 INFO:tasks.rados.rados.0.smithi017.stdout:1801: seq_num 868 ranges {588476=580374,1643429=603638} 2023-03-29T14:09:44.274 INFO:tasks.rados.rados.0.smithi017.stdout:1801: writing smithi017107884-9 from 588476 to 1168850 tid 1 2023-03-29T14:09:44.281 INFO:tasks.rados.rados.0.smithi017.stdout:1801: writing smithi017107884-9 from 1643429 to 2247067 tid 2 2023-03-29T14:09:44.281 INFO:tasks.rados.rados.0.smithi017.stdout:1802: delete oid 4 current snap is 207 2023-03-29T14:09:44.338 INFO:tasks.rados.rados.0.smithi017.stdout:1801: finishing write tid 1 to smithi017107884-9 2023-03-29T14:09:44.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:44.380 INFO:tasks.rados.rados.0.smithi017.stdout:1801: finishing write tid 2 to smithi017107884-9 2023-03-29T14:09:44.381 INFO:tasks.rados.rados.0.smithi017.stdout:1801: finishing write tid 3 to smithi017107884-9 2023-03-29T14:09:44.382 INFO:tasks.rados.rados.0.smithi017.stdout:1802: done (1 left) 2023-03-29T14:09:44.382 INFO:tasks.rados.rados.0.smithi017.stdout:1803: read oid 30 snap -1 2023-03-29T14:09:44.383 INFO:tasks.rados.rados.0.smithi017.stdout:1803: expect (ObjNum 853 snap 204 seq_num 853) 2023-03-29T14:09:44.383 INFO:tasks.rados.rados.0.smithi017.stdout:1804: read oid 10 snap -1 2023-03-29T14:09:44.383 INFO:tasks.rados.rados.0.smithi017.stdout:1804: expect deleted 2023-03-29T14:09:44.383 INFO:tasks.rados.rados.0.smithi017.stdout:1805: write (excl) oid 31 current snap is 207 2023-03-29T14:09:44.383 INFO:tasks.rados.rados.0.smithi017.stdout:1805: seq_num 870 ranges {622090=730872,1760393=607307} 2023-03-29T14:09:44.390 INFO:tasks.rados.rados.0.smithi017.stdout:1805: writing smithi017107884-31 from 622090 to 1352962 tid 1 2023-03-29T14:09:44.396 INFO:tasks.rados.rados.0.smithi017.stdout:1805: writing smithi017107884-31 from 1760393 to 2367700 tid 2 2023-03-29T14:09:44.396 INFO:tasks.rados.rados.0.smithi017.stdout:1806: write oid 33 current snap is 207 2023-03-29T14:09:44.396 INFO:tasks.rados.rados.0.smithi017.stdout:1801: finishing write tid 4 to smithi017107884-9 2023-03-29T14:09:44.396 INFO:tasks.rados.rados.0.smithi017.stdout:1801: oid 9 updating version 0 to 295 2023-03-29T14:09:44.396 INFO:tasks.rados.rados.0.smithi017.stdout:1801: oid 9 version 295 is already newer than 295 2023-03-29T14:09:44.396 INFO:tasks.rados.rados.0.smithi017.stdout:1801: oid 9 version 295 is already newer than 295 2023-03-29T14:09:44.397 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 9 v 295 (ObjNum 868 snap 207 seq_num 868) dirty exists 2023-03-29T14:09:44.397 INFO:tasks.rados.rados.0.smithi017.stdout:1801: left oid 9 (ObjNum 868 snap 207 seq_num 868) 2023-03-29T14:09:44.397 INFO:tasks.rados.rados.0.smithi017.stdout:1806: seq_num 871 ranges {638897=406121,1818875=750965} 2023-03-29T14:09:44.401 INFO:tasks.rados.rados.0.smithi017.stdout:1806: writing smithi017107884-33 from 638897 to 1045018 tid 1 2023-03-29T14:09:44.410 INFO:tasks.rados.rados.0.smithi017.stdout:1806: writing smithi017107884-33 from 1818875 to 2569840 tid 2 2023-03-29T14:09:44.410 INFO:tasks.rados.rados.0.smithi017.stdout:1801: done (4 left) 2023-03-29T14:09:44.410 INFO:tasks.rados.rados.0.smithi017.stdout:1804: done (3 left) 2023-03-29T14:09:44.410 INFO:tasks.rados.rados.0.smithi017.stdout:1807: write oid 21 current snap is 207 2023-03-29T14:09:44.410 INFO:tasks.rados.rados.0.smithi017.stdout:1807: seq_num 872 ranges {655704=481370,1761004=494623} 2023-03-29T14:09:44.416 INFO:tasks.rados.rados.0.smithi017.stdout:1807: writing smithi017107884-21 from 655704 to 1137074 tid 1 2023-03-29T14:09:44.423 INFO:tasks.rados.rados.0.smithi017.stdout:1807: writing smithi017107884-21 from 1761004 to 2255627 tid 2 2023-03-29T14:09:44.423 INFO:tasks.rados.rados.0.smithi017.stdout:1808: write (excl) oid 32 current snap is 207 2023-03-29T14:09:44.423 INFO:tasks.rados.rados.0.smithi017.stdout:1808: seq_num 873 ranges {672511=556619,1819486=754634} 2023-03-29T14:09:44.429 INFO:tasks.rados.rados.0.smithi017.stdout:1808: writing smithi017107884-32 from 672511 to 1229130 tid 1 2023-03-29T14:09:44.437 INFO:tasks.rados.rados.0.smithi017.stdout:1808: writing smithi017107884-32 from 1819486 to 2574120 tid 2 2023-03-29T14:09:44.437 INFO:tasks.rados.rados.0.smithi017.stdout:1809: delete oid 8 current snap is 207 2023-03-29T14:09:44.437 INFO:tasks.rados.rados.0.smithi017.stdout:1805: finishing write tid 1 to smithi017107884-31 2023-03-29T14:09:44.471 INFO:tasks.rados.rados.0.smithi017.stdout:1807: finishing write tid 1 to smithi017107884-21 2023-03-29T14:09:44.471 INFO:tasks.rados.rados.0.smithi017.stdout:1805: finishing write tid 2 to smithi017107884-31 2023-03-29T14:09:44.471 INFO:tasks.rados.rados.0.smithi017.stdout:1805: finishing write tid 3 to smithi017107884-31 2023-03-29T14:09:44.471 INFO:tasks.rados.rados.0.smithi017.stdout:1805: finishing write tid 4 to smithi017107884-31 2023-03-29T14:09:44.471 INFO:tasks.rados.rados.0.smithi017.stdout:1805: oid 31 updating version 0 to 171 2023-03-29T14:09:44.471 INFO:tasks.rados.rados.0.smithi017.stdout:1805: oid 31 version 171 is already newer than 171 2023-03-29T14:09:44.472 INFO:tasks.rados.rados.0.smithi017.stdout:1805: oid 31 version 171 is already newer than 169 2023-03-29T14:09:44.472 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 171 (ObjNum 870 snap 207 seq_num 870) dirty exists 2023-03-29T14:09:44.472 INFO:tasks.rados.rados.0.smithi017.stdout:1805: left oid 31 (ObjNum 870 snap 207 seq_num 870) 2023-03-29T14:09:44.501 INFO:tasks.rados.rados.0.smithi017.stdout:1807: finishing write tid 2 to smithi017107884-21 2023-03-29T14:09:44.502 INFO:tasks.rados.rados.0.smithi017.stdout:1807: finishing write tid 3 to smithi017107884-21 2023-03-29T14:09:44.502 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:44.501+0000 7f0d2353c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:44.503 INFO:tasks.rados.rados.0.smithi017.stdout:1807: finishing write tid 4 to smithi017107884-21 2023-03-29T14:09:44.503 INFO:tasks.rados.rados.0.smithi017.stdout:1807: oid 21 updating version 0 to 220 2023-03-29T14:09:44.503 INFO:tasks.rados.rados.0.smithi017.stdout:1807: oid 21 version 220 is already newer than 220 2023-03-29T14:09:44.503 INFO:tasks.rados.rados.0.smithi017.stdout:1807: oid 21 version 220 is already newer than 217 2023-03-29T14:09:44.503 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 21 v 220 (ObjNum 872 snap 207 seq_num 872) dirty exists 2023-03-29T14:09:44.503 INFO:tasks.rados.rados.0.smithi017.stdout:1807: left oid 21 (ObjNum 872 snap 207 seq_num 872) 2023-03-29T14:09:44.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:44.506+0000 7f0d2353c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:44.516 INFO:tasks.rados.rados.0.smithi017.stdout:1806: finishing write tid 1 to smithi017107884-33 2023-03-29T14:09:44.522 INFO:tasks.rados.rados.0.smithi017.stdout:1803: expect (ObjNum 853 snap 204 seq_num 853) 2023-03-29T14:09:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:44.664 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:44.664 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:44.724 INFO:tasks.rados.rados.0.smithi017.stdout:1803: done (5 left) 2023-03-29T14:09:44.725 INFO:tasks.rados.rados.0.smithi017.stdout:1805: done (4 left) 2023-03-29T14:09:44.725 INFO:tasks.rados.rados.0.smithi017.stdout:1807: done (3 left) 2023-03-29T14:09:44.725 INFO:tasks.rados.rados.0.smithi017.stdout:1809: done (2 left) 2023-03-29T14:09:44.725 INFO:tasks.rados.rados.0.smithi017.stdout:1810: snap_remove snap 205 2023-03-29T14:09:45.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~275s 2023-03-29T14:09:45.325 INFO:tasks.rados.rados.0.smithi017.stdout:1810: done (2 left) 2023-03-29T14:09:45.325 INFO:tasks.rados.rados.0.smithi017.stdout:1811: snap_create 2023-03-29T14:09:45.325 INFO:tasks.rados.rados.0.smithi017.stdout:1806: finishing write tid 2 to smithi017107884-33 2023-03-29T14:09:45.325 INFO:tasks.rados.rados.0.smithi017.stdout:1806: finishing write tid 3 to smithi017107884-33 2023-03-29T14:09:45.325 INFO:tasks.rados.rados.0.smithi017.stdout:1806: finishing write tid 4 to smithi017107884-33 2023-03-29T14:09:45.325 INFO:tasks.rados.rados.0.smithi017.stdout:1806: oid 33 updating version 0 to 245 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1806: oid 33 version 245 is already newer than 245 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1806: oid 33 version 245 is already newer than 245 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 245 (ObjNum 871 snap 207 seq_num 871) dirty exists 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1806: left oid 33 (ObjNum 871 snap 207 seq_num 871) 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1808: finishing write tid 1 to smithi017107884-32 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1808: finishing write tid 2 to smithi017107884-32 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1808: finishing write tid 3 to smithi017107884-32 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1808: finishing write tid 4 to smithi017107884-32 2023-03-29T14:09:45.326 INFO:tasks.rados.rados.0.smithi017.stdout:1808: oid 32 updating version 0 to 383 2023-03-29T14:09:45.327 INFO:tasks.rados.rados.0.smithi017.stdout:1808: oid 32 version 383 is already newer than 383 2023-03-29T14:09:45.327 INFO:tasks.rados.rados.0.smithi017.stdout:1808: oid 32 version 383 is already newer than 380 2023-03-29T14:09:45.327 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 32 v 383 (ObjNum 873 snap 207 seq_num 873) dirty exists 2023-03-29T14:09:45.327 INFO:tasks.rados.rados.0.smithi017.stdout:1808: left oid 32 (ObjNum 873 snap 207 seq_num 873) 2023-03-29T14:09:46.328 INFO:tasks.rados.rados.0.smithi017.stdout:1806: done (2 left) 2023-03-29T14:09:46.328 INFO:tasks.rados.rados.0.smithi017.stdout:1808: done (1 left) 2023-03-29T14:09:46.328 INFO:tasks.rados.rados.0.smithi017.stdout:1811: done (0 left) 2023-03-29T14:09:46.328 INFO:tasks.rados.rados.0.smithi017.stdout:1812: read oid 41 snap -1 2023-03-29T14:09:46.328 INFO:tasks.rados.rados.0.smithi017.stdout:1812: expect deleted 2023-03-29T14:09:46.329 INFO:tasks.rados.rados.0.smithi017.stdout:1813: snap_remove snap 206 2023-03-29T14:09:46.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:46.795 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:46.794+0000 7fbe915b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:46.798 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:46.798+0000 7fbe915b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:46.861 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:46.864 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:46.865 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:47.329 INFO:tasks.rados.rados.0.smithi017.stdout:1813: done (1 left) 2023-03-29T14:09:47.329 INFO:tasks.rados.rados.0.smithi017.stdout:1814: delete oid 31 current snap is 208 2023-03-29T14:09:47.403 INFO:tasks.rados.rados.0.smithi017.stdout:1812: done (1 left) 2023-03-29T14:09:47.403 INFO:tasks.rados.rados.0.smithi017.stdout:1814: done (0 left) 2023-03-29T14:09:47.403 INFO:tasks.rados.rados.0.smithi017.stdout:1815: snap_create 2023-03-29T14:09:48.334 INFO:tasks.rados.rados.0.smithi017.stdout:1815: done (0 left) 2023-03-29T14:09:48.334 INFO:tasks.rados.rados.0.smithi017.stdout:1816: delete oid 2 current snap is 209 2023-03-29T14:09:48.396 INFO:tasks.rados.rados.0.smithi017.stdout:1816: done (0 left) 2023-03-29T14:09:48.396 INFO:tasks.rados.rados.0.smithi017.stdout:1817: snap_remove snap 207 2023-03-29T14:09:48.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:48.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:48.994+0000 7f3e0a24e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:49.000 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:48.999+0000 7f3e0a24e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:49.063 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:49.066 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:49.066 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:49.126 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:49.126 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:49.126 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:49.254+0000 7f9b3baf7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:49.260+0000 7f9b3baf7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:49.339 INFO:tasks.rados.rados.0.smithi017.stdout:1817: done (0 left) 2023-03-29T14:09:49.339 INFO:tasks.rados.rados.0.smithi017.stdout:1818: read oid 49 snap -1 2023-03-29T14:09:49.339 INFO:tasks.rados.rados.0.smithi017.stdout:1818: expect (ObjNum 842 snap 203 seq_num 842) 2023-03-29T14:09:49.339 INFO:tasks.rados.rados.0.smithi017.stdout:1819: read oid 46 snap -1 2023-03-29T14:09:49.339 INFO:tasks.rados.rados.0.smithi017.stdout:1819: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:49.339 INFO:tasks.rados.rados.0.smithi017.stdout:1820: write oid 35 current snap is 209 2023-03-29T14:09:49.339 INFO:tasks.rados.rados.0.smithi017.stdout:1820: seq_num 877 ranges {739739=573968,2053414=686325} 2023-03-29T14:09:49.346 INFO:tasks.rados.rados.0.smithi017.stdout:1820: writing smithi017107884-35 from 739739 to 1313707 tid 1 2023-03-29T14:09:49.355 INFO:tasks.rados.rados.0.smithi017.stdout:1820: writing smithi017107884-35 from 2053414 to 2739739 tid 2 2023-03-29T14:09:49.355 INFO:tasks.rados.rados.0.smithi017.stdout:1821: write oid 23 current snap is 209 2023-03-29T14:09:49.355 INFO:tasks.rados.rados.0.smithi017.stdout:1821: seq_num 878 ranges {756546=649217,2111896=505630} 2023-03-29T14:09:49.362 INFO:tasks.rados.rados.0.smithi017.stdout:1821: writing smithi017107884-23 from 756546 to 1405763 tid 1 2023-03-29T14:09:49.369 INFO:tasks.rados.rados.0.smithi017.stdout:1821: writing smithi017107884-23 from 2111896 to 2617526 tid 2 2023-03-29T14:09:49.369 INFO:tasks.rados.rados.0.smithi017.stdout:1822: delete oid 3 current snap is 209 2023-03-29T14:09:49.449 INFO:tasks.rados.rados.0.smithi017.stdout:1818: expect (ObjNum 842 snap 203 seq_num 842) 2023-03-29T14:09:49.469 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:49.471 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:49.483 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":667,"stamp":"2023-03-29T14:09:49.337437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":202171976,"num_objects":1028,"num_object_clones":932,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":560,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":28,"num_read":4094,"num_read_kb":2424228,"num_write":5616,"num_write_kb":1560264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2973,"ondisk_log_size":2973,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1943052,"kb_used_data":1856684,"kb_used_omap":0,"kb_used_meta":86336,"kb_avail":375544308,"statfs":{"total":386547056640,"available":384557371392,"internally_reserved":0,"allocated":1901244416,"data_stored":2361506953,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":88408064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6723988,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-4,"num_read":59,"num_read_kb":33162,"num_write":88,"num_write_kb":26790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014103"},"pg_stats":[{"pgid":"3.f","version":"432'133","reported_seq":437,"reported_epoch":439,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:45.532902+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:45.532902+0000","last_peered":"2023-03-29T14:09:45.532902+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:45.532902+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202608,"num_objects":30,"num_object_clones":30,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65947,"num_write":180,"num_write_kb":53646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":461,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:43.339680+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:43.339680+0000","last_peered":"2023-03-29T14:09:43.339680+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:43.339680+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"437'268","reported_seq":974,"reported_epoch":439,"state":"active+clean","last_fresh":"2023-03-29T14:09:45.530715+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:45.530715+0000","last_peered":"2023-03-29T14:09:45.530715+0000","last_clean":"2023-03-29T14:09:45.530715+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:45.530715+0000","last_undegraded":"2023-03-29T14:09:45.530715+0000","last_fullsized":"2023-03-29T14:09:45.530715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16808056,"num_objects":100,"num_object_clones":90,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":409,"num_read_kb":289765,"num_write":512,"num_write_kb":152358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"437'295","reported_seq":1015,"reported_epoch":439,"state":"active+clean","last_fresh":"2023-03-29T14:09:45.524653+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:45.524653+0000","last_peered":"2023-03-29T14:09:45.524653+0000","last_clean":"2023-03-29T14:09:45.524653+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:45.524653+0000","last_undegraded":"2023-03-29T14:09:45.524653+0000","last_fullsized":"2023-03-29T14:09:45.524653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15127422,"num_objects":118,"num_object_clones":110,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":404,"num_read_kb":238712,"num_write":578,"num_write_kb":162888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"434'231","reported_seq":865,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-03-29T14:09:43.354299+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:43.354299+0000","last_peered":"2023-03-29T14:09:43.354299+0000","last_clean":"2023-03-29T14:09:43.354299+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:43.354299+0000","last_undegraded":"2023-03-29T14:09:43.354299+0000","last_fullsized":"2023-03-29T14:09:43.354299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15396202,"num_objects":82,"num_object_clones":74,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":347,"num_read_kb":229046,"num_write":460,"num_write_kb":135804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":371,"reported_epoch":439,"state":"active+undersized","last_fresh":"2023-03-29T14:09:45.525185+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:45.525185+0000","last_peered":"2023-03-29T14:09:45.525185+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:45.525185+0000","last_undegraded":"2023-03-29T14:09:45.525185+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"437'171","reported_seq":719,"reported_epoch":439,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:45.529951+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:45.529951+0000","last_peered":"2023-03-29T14:09:45.529951+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:45.529951+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10353904,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":178,"num_read_kb":74219,"num_write":310,"num_write_kb":87630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":60}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":594,"reported_epoch":439,"state":"active+clean","last_fresh":"2023-03-29T14:09:45.515815+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:45.515815+0000","last_peered":"2023-03-29T14:09:45.515815+0000","last_clean":"2023-03-29T14:09:45.515815+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:45.515815+0000","last_undegraded":"2023-03-29T14:09:45.515815+0000","last_fullsized":"2023-03-29T14:09:45.515815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":82092,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":340,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-03-29T14:09:43.355040+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:43.355040+0000","last_peered":"2023-03-29T14:09:43.355040+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:43.355040+0000","last_undegraded":"2023-03-29T14:09:43.355040+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"434'111","reported_seq":370,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:44.519561+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:44.519561+0000","last_peered":"2023-03-29T14:09:44.519561+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:44.519561+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9580782,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":46624,"num_write":146,"num_write_kb":39178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":374,"reported_epoch":439,"state":"active+undersized","last_fresh":"2023-03-29T14:09:45.533860+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:45.533860+0000","last_peered":"2023-03-29T14:09:45.533860+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:45.533860+0000","last_undegraded":"2023-03-29T14:09:45.533860+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"434'123","reported_seq":626,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:44.409540+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:44.409540+0000","last_peered":"2023-03-29T14:09:44.409540+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:44.409540+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9614396,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":107609,"num_write":246,"num_write_kb":72486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":201,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-03-29T14:09:43.345104+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:43.345104+0000","last_peered":"2023-03-29T14:09:43.345104+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:43.345104+0000","last_undegraded":"2023-03-29T14:09:43.345104+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":708,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:43.334619+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:43.334619+0000","last_peered":"2023-03-29T14:09:43.334619+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:43.334619+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":374,"reported_epoch":439,"state":"active+undersized","last_fresh":"2023-03-29T14:09:45.521611+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:45.521611+0000","last_peered":"2023-03-29T14:09:45.521611+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:45.521611+0000","last_undegraded":"2023-03-29T14:09:45.521611+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":565,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:43.327679+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:43.327679+0000","last_peered":"2023-03-29T14:09:43.327679+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:43.327679+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"437'220","reported_seq":905,"reported_epoch":439,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:45.531405+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:45.531405+0000","last_peered":"2023-03-29T14:09:45.531405+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:45.531405+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17077034,"num_objects":82,"num_object_clones":74,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":391,"num_read_kb":264973,"num_write":428,"num_write_kb":123558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":82}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":374,"reported_epoch":439,"state":"active+undersized","last_fresh":"2023-03-29T14:09:45.518200+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:45.518200+0000","last_peered":"2023-03-29T14:09:45.518200+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:45.518200+0000","last_undegraded":"2023-03-29T14:09:45.518200+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"437'383","reported_seq":1207,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:44.628022+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:44.628022+0000","last_peered":"2023-03-29T14:09:44.628022+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:44.628022+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20674128,"num_objects":140,"num_object_clones":126,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":570,"num_read_kb":350517,"num_write":714,"num_write_kb":208938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":340,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-03-29T14:09:43.302357+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:43.302357+0000","last_peered":"2023-03-29T14:09:43.302357+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:43.302357+0000","last_undegraded":"2023-03-29T14:09:43.302357+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"434'124","reported_seq":608,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-03-29T14:09:43.362037+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:43.362037+0000","last_peered":"2023-03-29T14:09:43.362037+0000","last_clean":"2023-03-29T14:09:43.362037+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:43.362037+0000","last_undegraded":"2023-03-29T14:09:43.362037+0000","last_fullsized":"2023-03-29T14:09:43.362037+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362258,"num_objects":44,"num_object_clones":40,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":162,"num_read_kb":97291,"num_write":252,"num_write_kb":73624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"426'246","reported_seq":854,"reported_epoch":438,"state":"active+clean","last_fresh":"2023-03-29T14:09:43.346963+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:43.346963+0000","last_peered":"2023-03-29T14:09:43.346963+0000","last_clean":"2023-03-29T14:09:43.346963+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:43.346963+0000","last_undegraded":"2023-03-29T14:09:43.346963+0000","last_fullsized":"2023-03-29T14:09:43.346963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278190,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":306,"num_read_kb":177815,"num_write":438,"num_write_kb":126954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":340,"reported_epoch":438,"state":"active+undersized","last_fresh":"2023-03-29T14:09:43.342675+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:43.342675+0000","last_peered":"2023-03-29T14:09:43.342675+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:43.342675+0000","last_undegraded":"2023-03-29T14:09:43.342675+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":374,"reported_epoch":439,"state":"active+undersized","last_fresh":"2023-03-29T14:09:45.526998+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:45.526998+0000","last_peered":"2023-03-29T14:09:45.526998+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:45.526998+0000","last_undegraded":"2023-03-29T14:09:45.526998+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"437'247","reported_seq":895,"reported_epoch":438,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:44.594474+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:44.594474+0000","last_peered":"2023-03-29T14:09:44.594474+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:44.594474+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20707280,"num_objects":90,"num_object_clones":82,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":342,"num_read_kb":213660,"num_write":468,"num_write_kb":140032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":90}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":200991240,"num_objects":1024,"num_object_clones":932,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":28,"num_read":3952,"num_read_kb":2424108,"num_write":5342,"num_write_kb":1556476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2876,"ondisk_log_size":2876,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706124,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":573632,"kb_used_data":549496,"kb_used_omap":0,"kb_used_meta":24128,"kb_avail":93798208,"statfs":{"total":96636764160,"available":96049364992,"internally_reserved":0,"allocated":562683904,"data_stored":701090376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24707072},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":635944,"kb_used_data":610400,"kb_used_omap":0,"kb_used_meta":25536,"kb_avail":93735896,"statfs":{"total":96636764160,"available":95985557504,"internally_reserved":0,"allocated":625049600,"data_stored":780143947,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":26148864},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":577532,"kb_used_data":552948,"kb_used_omap":0,"kb_used_meta":24576,"kb_avail":93794308,"statfs":{"total":96636764160,"available":96045371392,"internally_reserved":0,"allocated":566218752,"data_stored":707040405,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25165824},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:49.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.e 3.3 1.0 3.2 3.1 2023-03-29T14:09:49.626 INFO:tasks.rados.rados.0.smithi017.stdout:1818: done (4 left) 2023-03-29T14:09:49.660 INFO:tasks.rados.rados.0.smithi017.stdout:1822: done (3 left) 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1823: snap_create 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: finishing write tid 1 to smithi017107884-23 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: finishing write tid 2 to smithi017107884-23 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: finishing write tid 3 to smithi017107884-23 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: finishing write tid 4 to smithi017107884-23 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: oid 23 updating version 0 to 388 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: oid 23 version 388 is already newer than 388 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: oid 23 version 388 is already newer than 388 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 23 v 388 (ObjNum 878 snap 209 seq_num 878) dirty exists 2023-03-29T14:09:49.661 INFO:tasks.rados.rados.0.smithi017.stdout:1821: left oid 23 (ObjNum 878 snap 209 seq_num 878) 2023-03-29T14:09:49.662 INFO:tasks.rados.rados.0.smithi017.stdout:1820: finishing write tid 1 to smithi017107884-35 2023-03-29T14:09:49.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:49.624+0000 7fafb85f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:49.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:49.638+0000 7fafb85f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:49.662 INFO:tasks.rados.rados.0.smithi017.stdout:1819: expect (ObjNum 651 snap 155 seq_num 651) 2023-03-29T14:09:49.760 INFO:tasks.rados.rados.0.smithi017.stdout:1820: finishing write tid 2 to smithi017107884-35 2023-03-29T14:09:49.761 INFO:tasks.rados.rados.0.smithi017.stdout:1820: finishing write tid 3 to smithi017107884-35 2023-03-29T14:09:49.761 INFO:tasks.rados.rados.0.smithi017.stdout:1820: finishing write tid 4 to smithi017107884-35 2023-03-29T14:09:49.761 INFO:tasks.rados.rados.0.smithi017.stdout:1820: oid 35 updating version 0 to 127 2023-03-29T14:09:49.761 INFO:tasks.rados.rados.0.smithi017.stdout:1820: oid 35 updating version 127 to 128 2023-03-29T14:09:49.761 INFO:tasks.rados.rados.0.smithi017.stdout:1820: oid 35 version 128 is already newer than 128 2023-03-29T14:09:49.761 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 35 v 128 (ObjNum 877 snap 209 seq_num 877) dirty exists 2023-03-29T14:09:49.761 INFO:tasks.rados.rados.0.smithi017.stdout:1820: left oid 35 (ObjNum 877 snap 209 seq_num 877) 2023-03-29T14:09:49.848 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3] on osd.0 to force-backfill; instructing pg(s) [1.0,3.e] on osd.1 to force-backfill; instructing pg(s) [3.2] on osd.3 to force-backfill; 2023-03-29T14:09:49.849 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:09:50.342 INFO:tasks.rados.rados.0.smithi017.stdout:1819: done (3 left) 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1820: done (2 left) 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1821: done (1 left) 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1823: done (0 left) 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1824: read oid 33 snap -1 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1824: expect (ObjNum 871 snap 207 seq_num 871) 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1825: read oid 13 snap -1 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1825: expect deleted 2023-03-29T14:09:50.343 INFO:tasks.rados.rados.0.smithi017.stdout:1826: delete oid 10 current snap is 210 2023-03-29T14:09:50.383 INFO:tasks.rados.rados.0.smithi017.stdout:1826: done (2 left) 2023-03-29T14:09:50.383 INFO:tasks.rados.rados.0.smithi017.stdout:1827: write oid 1 current snap is 210 2023-03-29T14:09:50.383 INFO:tasks.rados.rados.0.smithi017.stdout:1827: seq_num 881 ranges {406967=474964,1370989=769310,2677186=129781} 2023-03-29T14:09:50.388 INFO:tasks.rados.rados.0.smithi017.stdout:1827: writing smithi017107884-1 from 406967 to 881931 tid 1 2023-03-29T14:09:50.395 INFO:tasks.rados.rados.0.smithi017.stdout:1827: writing smithi017107884-1 from 1370989 to 2140299 tid 2 2023-03-29T14:09:50.399 INFO:tasks.rados.rados.0.smithi017.stdout:1827: writing smithi017107884-1 from 2677186 to 2806967 tid 3 2023-03-29T14:09:50.399 INFO:tasks.rados.rados.0.smithi017.stdout:1828: snap_create 2023-03-29T14:09:50.437 INFO:tasks.rados.rados.0.smithi017.stdout:1824: expect (ObjNum 871 snap 207 seq_num 871) 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: finishing write tid 1 to smithi017107884-1 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: finishing write tid 2 to smithi017107884-1 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: finishing write tid 3 to smithi017107884-1 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: finishing write tid 4 to smithi017107884-1 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: finishing write tid 5 to smithi017107884-1 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: oid 1 updating version 0 to 252 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: oid 1 version 252 is already newer than 252 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: oid 1 version 252 is already newer than 252 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:1827: oid 1 version 252 is already newer than 252 2023-03-29T14:09:50.542 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 252 (ObjNum 881 snap 210 seq_num 881) dirty exists 2023-03-29T14:09:50.543 INFO:tasks.rados.rados.0.smithi017.stdout:1827: left oid 1 (ObjNum 881 snap 210 seq_num 881) 2023-03-29T14:09:50.771 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~281s 2023-03-29T14:09:51.067 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:51.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:51.195+0000 7fa84e0b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:51.199+0000 7fa84e0b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:51.264 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:51.265 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:51.345 INFO:tasks.rados.rados.0.smithi017.stdout:1824: done (3 left) 2023-03-29T14:09:51.345 INFO:tasks.rados.rados.0.smithi017.stdout:1825: done (2 left) 2023-03-29T14:09:51.346 INFO:tasks.rados.rados.0.smithi017.stdout:1827: done (1 left) 2023-03-29T14:09:51.346 INFO:tasks.rados.rados.0.smithi017.stdout:1828: done (0 left) 2023-03-29T14:09:51.346 INFO:tasks.rados.rados.0.smithi017.stdout:1829: write (excl) oid 26 current snap is 211 2023-03-29T14:09:51.346 INFO:tasks.rados.rados.0.smithi017.stdout:1829: seq_num 882 ranges {423774=666566,1545824=512968,2704609=119165} 2023-03-29T14:09:51.351 INFO:tasks.rados.rados.0.smithi017.stdout:1829: writing smithi017107884-26 from 423774 to 1090340 tid 1 2023-03-29T14:09:51.356 INFO:tasks.rados.rados.0.smithi017.stdout:1829: writing smithi017107884-26 from 1545824 to 2058792 tid 2 2023-03-29T14:09:51.361 INFO:tasks.rados.rados.0.smithi017.stdout:1829: writing smithi017107884-26 from 2704609 to 2823774 tid 3 2023-03-29T14:09:51.361 INFO:tasks.rados.rados.0.smithi017.stdout:1830: snap_remove snap 209 2023-03-29T14:09:52.349 INFO:tasks.rados.rados.0.smithi017.stdout:1830: done (1 left) 2023-03-29T14:09:52.349 INFO:tasks.rados.rados.0.smithi017.stdout:1831: snap_remove snap 210 2023-03-29T14:09:53.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1831: done (1 left) 2023-03-29T14:09:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1832: snap_create 2023-03-29T14:09:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1829: finishing write tid 1 to smithi017107884-26 2023-03-29T14:09:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1829: finishing write tid 2 to smithi017107884-26 2023-03-29T14:09:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1829: finishing write tid 3 to smithi017107884-26 2023-03-29T14:09:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1829: finishing write tid 4 to smithi017107884-26 2023-03-29T14:09:53.267 INFO:tasks.rados.rados.0.smithi017.stdout:1829: finishing write tid 5 to smithi017107884-26 2023-03-29T14:09:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1829: oid 26 updating version 0 to 225 2023-03-29T14:09:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1829: oid 26 version 225 is already newer than 225 2023-03-29T14:09:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1829: oid 26 version 225 is already newer than 221 2023-03-29T14:09:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1829: oid 26 version 225 is already newer than 225 2023-03-29T14:09:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 26 v 225 (ObjNum 882 snap 211 seq_num 882) dirty exists 2023-03-29T14:09:53.268 INFO:tasks.rados.rados.0.smithi017.stdout:1829: left oid 26 (ObjNum 882 snap 211 seq_num 882) 2023-03-29T14:09:53.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:53.396+0000 7f27b6054700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:53.400 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:53.400+0000 7f27b6054700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:53.463 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:53.465 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:53.466 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:54.270 INFO:tasks.rados.rados.0.smithi017.stdout:1829: done (1 left) 2023-03-29T14:09:54.270 INFO:tasks.rados.rados.0.smithi017.stdout:1832: done (0 left) 2023-03-29T14:09:54.270 INFO:tasks.rados.rados.0.smithi017.stdout:1833: write oid 8 current snap is 212 2023-03-29T14:09:54.270 INFO:tasks.rados.rados.0.smithi017.stdout:1833: seq_num 883 ranges {440581=741815,1604306=656626,2732032=108549} 2023-03-29T14:09:54.276 INFO:tasks.rados.rados.0.smithi017.stdout:1833: writing smithi017107884-8 from 440581 to 1182396 tid 1 2023-03-29T14:09:54.282 INFO:tasks.rados.rados.0.smithi017.stdout:1833: writing smithi017107884-8 from 1604306 to 2260932 tid 2 2023-03-29T14:09:54.285 INFO:tasks.rados.rados.0.smithi017.stdout:1833: writing smithi017107884-8 from 2732032 to 2840581 tid 3 2023-03-29T14:09:54.285 INFO:tasks.rados.rados.0.smithi017.stdout:1834: snap_create 2023-03-29T14:09:54.387 INFO:tasks.rados.rados.0.smithi017.stdout:1833: finishing write tid 1 to smithi017107884-8 2023-03-29T14:09:54.389 INFO:tasks.rados.rados.0.smithi017.stdout:1833: finishing write tid 2 to smithi017107884-8 2023-03-29T14:09:54.390 INFO:tasks.rados.rados.0.smithi017.stdout:1833: finishing write tid 3 to smithi017107884-8 2023-03-29T14:09:54.393 INFO:tasks.rados.rados.0.smithi017.stdout:1833: finishing write tid 4 to smithi017107884-8 2023-03-29T14:09:54.395 INFO:tasks.rados.rados.0.smithi017.stdout:1833: finishing write tid 5 to smithi017107884-8 2023-03-29T14:09:54.395 INFO:tasks.rados.rados.0.smithi017.stdout:1833: oid 8 updating version 0 to 256 2023-03-29T14:09:54.395 INFO:tasks.rados.rados.0.smithi017.stdout:1833: oid 8 version 256 is already newer than 256 2023-03-29T14:09:54.395 INFO:tasks.rados.rados.0.smithi017.stdout:1833: oid 8 version 256 is already newer than 256 2023-03-29T14:09:54.395 INFO:tasks.rados.rados.0.smithi017.stdout:1833: oid 8 version 256 is already newer than 256 2023-03-29T14:09:54.395 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 8 v 256 (ObjNum 883 snap 212 seq_num 883) dirty exists 2023-03-29T14:09:54.395 INFO:tasks.rados.rados.0.smithi017.stdout:1833: left oid 8 (ObjNum 883 snap 212 seq_num 883) 2023-03-29T14:09:54.860 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:09:54.861 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:09:54.861 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:09:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:54.989+0000 7f4d54dfc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:54.996+0000 7f4d54dfc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:09:55.207 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:09:55.219 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":675,"stamp":"2023-03-29T14:09:54.268396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186473710,"num_objects":1040,"num_object_clones":944,"num_object_copies":3120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":4123,"num_read_kb":2441328,"num_write":5652,"num_write_kb":1570210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2996,"ondisk_log_size":2996,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1952200,"kb_used_data":1865512,"kb_used_omap":0,"kb_used_meta":86656,"kb_avail":375535160,"statfs":{"total":386547056640,"available":384548003840,"internally_reserved":0,"allocated":1910284288,"data_stored":2375781221,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":88735744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15698266,"num_objects":12,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":29,"num_read_kb":17100,"num_write":36,"num_write_kb":9946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.927582"},"pg_stats":[{"pgid":"3.f","version":"432'133","reported_seq":441,"reported_epoch":444,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:50.372285+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:50.372285+0000","last_peered":"2023-03-29T14:09:50.372285+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:50.372285+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202608,"num_objects":30,"num_object_clones":30,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65947,"num_write":180,"num_write_kb":53646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":464,"reported_epoch":443,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:49.455896+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:49.455896+0000","last_peered":"2023-03-29T14:09:49.455896+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:49.455896+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"437'268","reported_seq":979,"reported_epoch":444,"state":"active+clean","last_fresh":"2023-03-29T14:09:50.383665+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:50.383665+0000","last_peered":"2023-03-29T14:09:50.383665+0000","last_clean":"2023-03-29T14:09:50.383665+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:50.383665+0000","last_undegraded":"2023-03-29T14:09:50.383665+0000","last_fullsized":"2023-03-29T14:09:50.383665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16808056,"num_objects":100,"num_object_clones":90,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":410,"num_read_kb":289765,"num_write":512,"num_write_kb":152358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"437'295","reported_seq":1019,"reported_epoch":444,"state":"active+clean","last_fresh":"2023-03-29T14:09:50.363432+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:50.363432+0000","last_peered":"2023-03-29T14:09:50.363432+0000","last_clean":"2023-03-29T14:09:50.363432+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:50.363432+0000","last_undegraded":"2023-03-29T14:09:50.363432+0000","last_fullsized":"2023-03-29T14:09:50.363432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15127422,"num_objects":118,"num_object_clones":110,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":404,"num_read_kb":238712,"num_write":578,"num_write_kb":162888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"434'231","reported_seq":871,"reported_epoch":443,"state":"active+clean","last_fresh":"2023-03-29T14:09:49.638666+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:49.638666+0000","last_peered":"2023-03-29T14:09:49.638666+0000","last_clean":"2023-03-29T14:09:49.638666+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:49.638666+0000","last_undegraded":"2023-03-29T14:09:49.638666+0000","last_fullsized":"2023-03-29T14:09:49.638666+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15396202,"num_objects":82,"num_object_clones":74,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":354,"num_read_kb":235839,"num_write":460,"num_write_kb":135804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":375,"reported_epoch":444,"state":"active+undersized","last_fresh":"2023-03-29T14:09:50.368238+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:50.368238+0000","last_peered":"2023-03-29T14:09:50.368238+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:50.368238+0000","last_undegraded":"2023-03-29T14:09:50.368238+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"440'173","reported_seq":725,"reported_epoch":444,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:50.380398+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:50.380398+0000","last_peered":"2023-03-29T14:09:50.380398+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:50.380398+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5109724,"num_objects":62,"num_object_clones":56,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":62,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":178,"num_read_kb":74219,"num_write":310,"num_write_kb":87630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":62}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"411'103","reported_seq":598,"reported_epoch":444,"state":"active+clean","last_fresh":"2023-03-29T14:09:50.371290+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:50.371290+0000","last_peered":"2023-03-29T14:09:50.371290+0000","last_clean":"2023-03-29T14:09:50.371290+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:50.371290+0000","last_undegraded":"2023-03-29T14:09:50.371290+0000","last_fullsized":"2023-03-29T14:09:50.371290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14353838,"num_objects":36,"num_object_clones":32,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":82092,"num_write":192,"num_write_kb":55016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":343,"reported_epoch":443,"state":"active+undersized","last_fresh":"2023-03-29T14:09:49.494837+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:49.494837+0000","last_peered":"2023-03-29T14:09:49.494837+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:49.494837+0000","last_undegraded":"2023-03-29T14:09:49.494837+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"434'111","reported_seq":375,"reported_epoch":443,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:49.388533+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:49.388533+0000","last_peered":"2023-03-29T14:09:49.388533+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:49.388533+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9580782,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":46624,"num_write":146,"num_write_kb":39178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":378,"reported_epoch":444,"state":"active+undersized","last_fresh":"2023-03-29T14:09:50.377046+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:50.377046+0000","last_peered":"2023-03-29T14:09:50.377046+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:50.377046+0000","last_undegraded":"2023-03-29T14:09:50.377046+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"434'123","reported_seq":631,"reported_epoch":443,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:49.383561+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:49.383561+0000","last_peered":"2023-03-29T14:09:49.383561+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:49.383561+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9614396,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":107609,"num_write":246,"num_write_kb":72486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":204,"reported_epoch":443,"state":"active+undersized","last_fresh":"2023-03-29T14:09:49.442129+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:49.442129+0000","last_peered":"2023-03-29T14:09:49.442129+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:49.442129+0000","last_undegraded":"2023-03-29T14:09:49.442129+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":711,"reported_epoch":443,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:49.485971+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:49.485971+0000","last_peered":"2023-03-29T14:09:49.485971+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:49.485971+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":378,"reported_epoch":444,"state":"active+undersized","last_fresh":"2023-03-29T14:09:50.364021+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:50.364021+0000","last_peered":"2023-03-29T14:09:50.364021+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:50.364021+0000","last_undegraded":"2023-03-29T14:09:50.364021+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":568,"reported_epoch":443,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:49.458962+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:49.458962+0000","last_peered":"2023-03-29T14:09:49.458962+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:49.458962+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"444'225","reported_seq":918,"reported_epoch":444,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:51.420562+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:51.420562+0000","last_peered":"2023-03-29T14:09:51.420562+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:51.420562+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14993032,"num_objects":84,"num_object_clones":76,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":394,"num_read_kb":264974,"num_write":438,"num_write_kb":126096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":84}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":378,"reported_epoch":444,"state":"active+undersized","last_fresh":"2023-03-29T14:09:50.375733+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:50.375733+0000","last_peered":"2023-03-29T14:09:50.375733+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:50.375733+0000","last_undegraded":"2023-03-29T14:09:50.375733+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"442'390","reported_seq":1225,"reported_epoch":443,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:49.534657+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:49.534657+0000","last_peered":"2023-03-29T14:09:49.534657+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:49.534657+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15161102,"num_objects":144,"num_object_clones":130,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":579,"num_read_kb":355983,"num_write":722,"num_write_kb":211196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":343,"reported_epoch":443,"state":"active+undersized","last_fresh":"2023-03-29T14:09:49.425817+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:49.425817+0000","last_peered":"2023-03-29T14:09:49.425817+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:49.425817+0000","last_undegraded":"2023-03-29T14:09:49.425817+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"442'128","reported_seq":618,"reported_epoch":443,"state":"active+clean","last_fresh":"2023-03-29T14:09:49.688318+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:49.688318+0000","last_peered":"2023-03-29T14:09:49.688318+0000","last_clean":"2023-03-29T14:09:49.688318+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:49.688318+0000","last_undegraded":"2023-03-29T14:09:49.688318+0000","last_fullsized":"2023-03-29T14:09:49.688318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10421132,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":97292,"num_write":260,"num_write_kb":76088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"426'246","reported_seq":857,"reported_epoch":443,"state":"active+clean","last_fresh":"2023-03-29T14:09:49.475122+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:49.475122+0000","last_peered":"2023-03-29T14:09:49.475122+0000","last_clean":"2023-03-29T14:09:49.475122+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:49.475122+0000","last_undegraded":"2023-03-29T14:09:49.475122+0000","last_fullsized":"2023-03-29T14:09:49.475122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278190,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":306,"num_read_kb":177815,"num_write":438,"num_write_kb":126954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":343,"reported_epoch":443,"state":"active+undersized","last_fresh":"2023-03-29T14:09:49.509141+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:49.509141+0000","last_peered":"2023-03-29T14:09:49.509141+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:49.509141+0000","last_undegraded":"2023-03-29T14:09:49.509141+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":378,"reported_epoch":444,"state":"active+undersized","last_fresh":"2023-03-29T14:09:50.378363+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:50.378363+0000","last_peered":"2023-03-29T14:09:50.378363+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:50.378363+0000","last_undegraded":"2023-03-29T14:09:50.378363+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"443'252","reported_seq":912,"reported_epoch":443,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:50.481936+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:50.481936+0000","last_peered":"2023-03-29T14:09:50.481936+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:50.481936+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18791348,"num_objects":92,"num_object_clones":84,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":92,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":350,"num_read_kb":218499,"num_write":478,"num_write_kb":142718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":92}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":185292974,"num_objects":1036,"num_object_clones":944,"num_object_copies":3108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":3981,"num_read_kb":2441208,"num_write":5378,"num_write_kb":1566422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2899,"ondisk_log_size":2899,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706129,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":576184,"kb_used_data":551920,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":93795656,"statfs":{"total":96636764160,"available":96046751744,"internally_reserved":0,"allocated":565166080,"data_stored":705374349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706128,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":639844,"kb_used_data":614172,"kb_used_omap":0,"kb_used_meta":25664,"kb_avail":93731996,"statfs":{"total":96636764160,"available":95981563904,"internally_reserved":0,"allocated":628912128,"data_stored":786107288,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":26279936},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706128,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":580228,"kb_used_data":555580,"kb_used_omap":0,"kb_used_meta":24640,"kb_avail":93791612,"statfs":{"total":96636764160,"available":96042610688,"internally_reserved":0,"allocated":568913920,"data_stored":711067359,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25231360},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:09:55.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.f 3.e 3.3 3.0 3.1 3.6 2023-03-29T14:09:55.272 INFO:tasks.rados.rados.0.smithi017.stdout:1833: done (1 left) 2023-03-29T14:09:55.272 INFO:tasks.rados.rados.0.smithi017.stdout:1834: done (0 left) 2023-03-29T14:09:55.272 INFO:tasks.rados.rados.0.smithi017.stdout:1835: delete oid 8 current snap is 213 2023-03-29T14:09:55.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:55.354+0000 7efd369f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:55.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:55.367+0000 7efd369f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:55.374 INFO:tasks.rados.rados.0.smithi017.stdout:1835: done (0 left) 2023-03-29T14:09:55.374 INFO:tasks.rados.rados.0.smithi017.stdout:1836: read oid 34 snap -1 2023-03-29T14:09:55.375 INFO:tasks.rados.rados.0.smithi017.stdout:1836: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:09:55.375 INFO:tasks.rados.rados.0.smithi017.stdout:1837: read oid 11 snap 208 2023-03-29T14:09:55.375 INFO:tasks.rados.rados.0.smithi017.stdout:1837: expect deleted 2023-03-29T14:09:55.375 INFO:tasks.rados.rados.0.smithi017.stdout:1838: snap_create 2023-03-29T14:09:55.395 INFO:tasks.rados.rados.0.smithi017.stdout:1836: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:09:55.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:55.577 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.1,3.3,3.6] on osd.0 to force-recovery; instructing pg(s) [3.0,3.e] on osd.1 to force-recovery; instructing pg(s) [3.f] on osd.3 to force-recovery; 2023-03-29T14:09:55.577 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:09:55.598 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:55.597+0000 7f5ac8daa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:55.601 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:55.601+0000 7f5ac8daa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:55.664 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:55.667 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:55.667 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:56.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~286s 2023-03-29T14:09:56.357 INFO:tasks.rados.rados.0.smithi017.stdout:1836: done (2 left) 2023-03-29T14:09:56.357 INFO:tasks.rados.rados.0.smithi017.stdout:1837: done (1 left) 2023-03-29T14:09:56.357 INFO:tasks.rados.rados.0.smithi017.stdout:1838: done (0 left) 2023-03-29T14:09:56.357 INFO:tasks.rados.rados.0.smithi017.stdout:1839: read oid 42 snap -1 2023-03-29T14:09:56.358 INFO:tasks.rados.rados.0.smithi017.stdout:1839: expect (ObjNum 839 snap 202 seq_num 839) 2023-03-29T14:09:56.358 INFO:tasks.rados.rados.0.smithi017.stdout:1840: read oid 37 snap -1 2023-03-29T14:09:56.358 INFO:tasks.rados.rados.0.smithi017.stdout:1840: expect (ObjNum 860 snap 205 seq_num 860) 2023-03-29T14:09:56.358 INFO:tasks.rados.rados.0.smithi017.stdout:1841: delete oid 18 current snap is 214 2023-03-29T14:09:56.400 INFO:tasks.rados.rados.0.smithi017.stdout:1841: done (2 left) 2023-03-29T14:09:56.400 INFO:tasks.rados.rados.0.smithi017.stdout:1842: read oid 18 snap -1 2023-03-29T14:09:56.400 INFO:tasks.rados.rados.0.smithi017.stdout:1842: expect deleted 2023-03-29T14:09:56.400 INFO:tasks.rados.rados.0.smithi017.stdout:1843: write (excl) oid 28 current snap is 214 2023-03-29T14:09:56.400 INFO:tasks.rados.rados.0.smithi017.stdout:1843: seq_num 886 ranges {491002=567562,1663399=520306,2814301=76701} 2023-03-29T14:09:56.406 INFO:tasks.rados.rados.0.smithi017.stdout:1843: writing smithi017107884-28 from 491002 to 1058564 tid 1 2023-03-29T14:09:56.412 INFO:tasks.rados.rados.0.smithi017.stdout:1843: writing smithi017107884-28 from 1663399 to 2183705 tid 2 2023-03-29T14:09:56.417 INFO:tasks.rados.rados.0.smithi017.stdout:1843: writing smithi017107884-28 from 2814301 to 2891002 tid 3 2023-03-29T14:09:56.417 INFO:tasks.rados.rados.0.smithi017.stdout:1844: write oid 14 current snap is 214 2023-03-29T14:09:56.417 INFO:tasks.rados.rados.0.smithi017.stdout:1844: seq_num 887 ranges {507809=642811,1721881=663964} 2023-03-29T14:09:56.423 INFO:tasks.rados.rados.0.smithi017.stdout:1844: writing smithi017107884-14 from 507809 to 1150620 tid 1 2023-03-29T14:09:56.431 INFO:tasks.rados.rados.0.smithi017.stdout:1844: writing smithi017107884-14 from 1721881 to 2385845 tid 2 2023-03-29T14:09:56.432 INFO:tasks.rados.rados.0.smithi017.stdout:1845: write (excl) oid 3 current snap is 214 2023-03-29T14:09:56.432 INFO:tasks.rados.rados.0.smithi017.stdout:1845: seq_num 888 ranges {524616=718060,1664010=523975,2752794=171822} 2023-03-29T14:09:56.440 INFO:tasks.rados.rados.0.smithi017.stdout:1845: writing smithi017107884-3 from 524616 to 1242676 tid 1 2023-03-29T14:09:56.446 INFO:tasks.rados.rados.0.smithi017.stdout:1845: writing smithi017107884-3 from 1664010 to 2187985 tid 2 2023-03-29T14:09:56.450 INFO:tasks.rados.rados.0.smithi017.stdout:1845: writing smithi017107884-3 from 2752794 to 2924616 tid 3 2023-03-29T14:09:56.450 INFO:tasks.rados.rados.0.smithi017.stdout:1846: read oid 38 snap -1 2023-03-29T14:09:56.451 INFO:tasks.rados.rados.0.smithi017.stdout:1846: expect (ObjNum 801 snap 195 seq_num 801) 2023-03-29T14:09:56.451 INFO:tasks.rados.rados.0.smithi017.stdout:1847: snap_remove snap 208 2023-03-29T14:09:57.360 INFO:tasks.rados.rados.0.smithi017.stdout:1847: done (7 left) 2023-03-29T14:09:57.360 INFO:tasks.rados.rados.0.smithi017.stdout:1848: delete oid 8 current snap is 214 2023-03-29T14:09:57.360 INFO:tasks.rados.rados.0.smithi017.stdout:1839: expect (ObjNum 839 snap 202 seq_num 839) 2023-03-29T14:09:57.491 INFO:tasks.rados.rados.0.smithi017.stdout:1839: done (7 left) 2023-03-29T14:09:57.491 INFO:tasks.rados.rados.0.smithi017.stdout:1842: done (6 left) 2023-03-29T14:09:57.491 INFO:tasks.rados.rados.0.smithi017.stdout:1848: done (5 left) 2023-03-29T14:09:57.491 INFO:tasks.rados.rados.0.smithi017.stdout:1849: snap_remove snap 212 2023-03-29T14:09:57.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:09:57.798 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:57.796+0000 7f7da623a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:57.801 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:57.801+0000 7f7da623a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:57.861 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:09:57.864 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:09:57.865 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:09:58.363 INFO:tasks.rados.rados.0.smithi017.stdout:1849: done (5 left) 2023-03-29T14:09:58.364 INFO:tasks.rados.rados.0.smithi017.stdout:1850: read oid 24 snap -1 2023-03-29T14:09:58.364 INFO:tasks.rados.rados.0.smithi017.stdout:1850: expect (ObjNum 847 snap 203 seq_num 847) 2023-03-29T14:09:58.364 INFO:tasks.rados.rados.0.smithi017.stdout:1851: delete oid 20 current snap is 214 2023-03-29T14:09:58.364 INFO:tasks.rados.rados.0.smithi017.stdout:1840: expect (ObjNum 860 snap 205 seq_num 860) 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1843: finishing write tid 1 to smithi017107884-28 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1844: finishing write tid 1 to smithi017107884-14 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1844: finishing write tid 2 to smithi017107884-14 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: finishing write tid 1 to smithi017107884-3 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: finishing write tid 2 to smithi017107884-3 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: finishing write tid 3 to smithi017107884-3 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: finishing write tid 4 to smithi017107884-3 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: finishing write tid 5 to smithi017107884-3 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: oid 3 updating version 0 to 391 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: oid 3 updating version 391 to 394 2023-03-29T14:09:58.461 INFO:tasks.rados.rados.0.smithi017.stdout:1845: oid 3 version 394 is already newer than 394 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1845: oid 3 version 394 is already newer than 394 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 3 v 394 (ObjNum 888 snap 214 seq_num 888) dirty exists 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1845: left oid 3 (ObjNum 888 snap 214 seq_num 888) 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1844: finishing write tid 3 to smithi017107884-14 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1844: finishing write tid 4 to smithi017107884-14 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1844: oid 14 updating version 0 to 178 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1844: oid 14 version 178 is already newer than 178 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1844: oid 14 version 178 is already newer than 178 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 14 v 178 (ObjNum 887 snap 214 seq_num 887) dirty exists 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1844: left oid 14 (ObjNum 887 snap 214 seq_num 887) 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: finishing write tid 2 to smithi017107884-28 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: finishing write tid 3 to smithi017107884-28 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: finishing write tid 4 to smithi017107884-28 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: finishing write tid 5 to smithi017107884-28 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: oid 28 updating version 0 to 108 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: oid 28 version 108 is already newer than 108 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: oid 28 version 108 is already newer than 108 2023-03-29T14:09:58.462 INFO:tasks.rados.rados.0.smithi017.stdout:1843: oid 28 version 108 is already newer than 104 2023-03-29T14:09:58.463 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 28 v 108 (ObjNum 886 snap 214 seq_num 886) dirty exists 2023-03-29T14:09:58.463 INFO:tasks.rados.rados.0.smithi017.stdout:1843: left oid 28 (ObjNum 886 snap 214 seq_num 886) 2023-03-29T14:09:58.463 INFO:tasks.rados.rados.0.smithi017.stdout:1846: expect (ObjNum 801 snap 195 seq_num 801) 2023-03-29T14:09:58.597 INFO:tasks.rados.rados.0.smithi017.stdout:1850: expect (ObjNum 847 snap 203 seq_num 847) 2023-03-29T14:09:58.759 INFO:tasks.rados.rados.0.smithi017.stdout:1840: done (6 left) 2023-03-29T14:09:58.759 INFO:tasks.rados.rados.0.smithi017.stdout:1843: done (5 left) 2023-03-29T14:09:58.759 INFO:tasks.rados.rados.0.smithi017.stdout:1844: done (4 left) 2023-03-29T14:09:58.759 INFO:tasks.rados.rados.0.smithi017.stdout:1845: done (3 left) 2023-03-29T14:09:58.759 INFO:tasks.rados.rados.0.smithi017.stdout:1846: done (2 left) 2023-03-29T14:09:58.759 INFO:tasks.rados.rados.0.smithi017.stdout:1850: done (1 left) 2023-03-29T14:09:58.760 INFO:tasks.rados.rados.0.smithi017.stdout:1851: done (0 left) 2023-03-29T14:09:58.760 INFO:tasks.rados.rados.0.smithi017.stdout:1852: delete oid 39 current snap is 214 2023-03-29T14:09:58.780 INFO:tasks.rados.rados.0.smithi017.stdout:1852: done (0 left) 2023-03-29T14:09:58.780 INFO:tasks.rados.rados.0.smithi017.stdout:1853: read oid 48 snap -1 2023-03-29T14:09:58.780 INFO:tasks.rados.rados.0.smithi017.stdout:1853: expect (ObjNum 776 snap 189 seq_num 776) 2023-03-29T14:09:58.780 INFO:tasks.rados.rados.0.smithi017.stdout:1854: snap_create 2023-03-29T14:09:58.794 INFO:tasks.rados.rados.0.smithi017.stdout:1853: expect (ObjNum 776 snap 189 seq_num 776) 2023-03-29T14:09:59.367 INFO:tasks.rados.rados.0.smithi017.stdout:1853: done (1 left) 2023-03-29T14:09:59.367 INFO:tasks.rados.rados.0.smithi017.stdout:1854: done (0 left) 2023-03-29T14:09:59.367 INFO:tasks.rados.rados.0.smithi017.stdout:1855: write (excl) oid 41 current snap is 215 2023-03-29T14:09:59.367 INFO:tasks.rados.rados.0.smithi017.stdout:1855: seq_num 892 ranges {591844=735409,1897938=531313,2862486=129358} 2023-03-29T14:09:59.374 INFO:tasks.rados.rados.0.smithi017.stdout:1855: writing smithi017107884-41 from 591844 to 1327253 tid 1 2023-03-29T14:09:59.380 INFO:tasks.rados.rados.0.smithi017.stdout:1855: writing smithi017107884-41 from 1897938 to 2429251 tid 2 2023-03-29T14:09:59.383 INFO:tasks.rados.rados.0.smithi017.stdout:1855: writing smithi017107884-41 from 2862486 to 2991844 tid 3 2023-03-29T14:09:59.383 INFO:tasks.rados.rados.0.smithi017.stdout:1856: read oid 39 snap -1 2023-03-29T14:09:59.384 INFO:tasks.rados.rados.0.smithi017.stdout:1856: expect deleted 2023-03-29T14:09:59.384 INFO:tasks.rados.rados.0.smithi017.stdout:1857: read oid 34 snap -1 2023-03-29T14:09:59.384 INFO:tasks.rados.rados.0.smithi017.stdout:1857: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:09:59.384 INFO:tasks.rados.rados.0.smithi017.stdout:1858: snap_remove snap 213 2023-03-29T14:09:59.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:09:59.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:59.993+0000 7f22bc22a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:09:59.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:09:59.997+0000 7f22bc22a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:10:00.064 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:10:00.064 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:10:00.371 INFO:tasks.rados.rados.0.smithi017.stdout:1858: done (3 left) 2023-03-29T14:10:00.371 INFO:tasks.rados.rados.0.smithi017.stdout:1859: snap_create 2023-03-29T14:10:00.371 INFO:tasks.rados.rados.0.smithi017.stdout:1857: expect (ObjNum 727 snap 177 seq_num 727) 2023-03-29T14:10:00.456 INFO:tasks.rados.rados.0.smithi017.stdout:1855: finishing write tid 1 to smithi017107884-41 2023-03-29T14:10:00.456 INFO:tasks.rados.rados.0.smithi017.stdout:1855: finishing write tid 2 to smithi017107884-41 2023-03-29T14:10:00.456 INFO:tasks.rados.rados.0.smithi017.stdout:1855: finishing write tid 3 to smithi017107884-41 2023-03-29T14:10:00.456 INFO:tasks.rados.rados.0.smithi017.stdout:1855: finishing write tid 4 to smithi017107884-41 2023-03-29T14:10:00.457 INFO:tasks.rados.rados.0.smithi017.stdout:1855: finishing write tid 5 to smithi017107884-41 2023-03-29T14:10:00.457 INFO:tasks.rados.rados.0.smithi017.stdout:1855: oid 41 updating version 0 to 250 2023-03-29T14:10:00.457 INFO:tasks.rados.rados.0.smithi017.stdout:1855: oid 41 version 250 is already newer than 250 2023-03-29T14:10:00.457 INFO:tasks.rados.rados.0.smithi017.stdout:1855: oid 41 version 250 is already newer than 247 2023-03-29T14:10:00.457 INFO:tasks.rados.rados.0.smithi017.stdout:1855: oid 41 version 250 is already newer than 250 2023-03-29T14:10:00.457 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 41 v 250 (ObjNum 892 snap 215 seq_num 892) dirty exists 2023-03-29T14:10:00.457 INFO:tasks.rados.rados.0.smithi017.stdout:1855: left oid 41 (ObjNum 892 snap 215 seq_num 892) 2023-03-29T14:10:00.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:10:00.590 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:10:00.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:10:00.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:00.720+0000 7fcddb78d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:00.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:00.727+0000 7fcddb78d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:00.933 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:10:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:10:00.947 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":684,"stamp":"2023-03-29T14:10:00.369493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185734268,"num_objects":1046,"num_object_clones":950,"num_object_copies":3138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":4143,"num_read_kb":2451790,"num_write":5680,"num_write_kb":1577990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3008,"ondisk_log_size":3008,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1957968,"kb_used_data":1871152,"kb_used_omap":0,"kb_used_meta":86784,"kb_avail":375529392,"statfs":{"total":386547056640,"available":384542097408,"internally_reserved":0,"allocated":1916059648,"data_stored":2383009858,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":88866816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-739442,"num_objects":4,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":5034,"num_write":18,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.014031"},"pg_stats":[{"pgid":"3.f","version":"432'133","reported_seq":445,"reported_epoch":449,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.507914+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:54.507914+0000","last_peered":"2023-03-29T14:09:54.507914+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:54.507914+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202608,"num_objects":30,"num_object_clones":30,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65947,"num_write":180,"num_write_kb":53646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":468,"reported_epoch":448,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.297020+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:09:54.297020+0000","last_peered":"2023-03-29T14:09:54.297020+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:09:54.297020+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":30742,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"437'268","reported_seq":983,"reported_epoch":449,"state":"active+clean","last_fresh":"2023-03-29T14:09:54.511358+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:54.511358+0000","last_peered":"2023-03-29T14:09:54.511358+0000","last_clean":"2023-03-29T14:09:54.511358+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:54.511358+0000","last_undegraded":"2023-03-29T14:09:54.511358+0000","last_fullsized":"2023-03-29T14:09:54.511358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16808056,"num_objects":100,"num_object_clones":90,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":410,"num_read_kb":289765,"num_write":512,"num_write_kb":152358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"437'295","reported_seq":1026,"reported_epoch":449,"state":"active+clean","last_fresh":"2023-03-29T14:09:56.404547+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:56.404547+0000","last_peered":"2023-03-29T14:09:56.404547+0000","last_clean":"2023-03-29T14:09:56.404547+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:56.404547+0000","last_undegraded":"2023-03-29T14:09:56.404547+0000","last_fullsized":"2023-03-29T14:09:56.404547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15127422,"num_objects":118,"num_object_clones":110,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":411,"num_read_kb":243746,"num_write":578,"num_write_kb":162888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"434'231","reported_seq":878,"reported_epoch":448,"state":"active+clean","last_fresh":"2023-03-29T14:09:55.389638+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:09:55.389638+0000","last_peered":"2023-03-29T14:09:55.389638+0000","last_clean":"2023-03-29T14:09:55.389638+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:09:55.389638+0000","last_undegraded":"2023-03-29T14:09:55.389638+0000","last_fullsized":"2023-03-29T14:09:55.389638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15396202,"num_objects":82,"num_object_clones":74,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":361,"num_read_kb":241266,"num_write":460,"num_write_kb":135804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":379,"reported_epoch":449,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.504043+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:54.504043+0000","last_peered":"2023-03-29T14:09:54.504043+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:54.504043+0000","last_undegraded":"2023-03-29T14:09:54.504043+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"449'177","reported_seq":736,"reported_epoch":449,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:56.452297+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:56.452297+0000","last_peered":"2023-03-29T14:09:56.452297+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:56.452297+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5815618,"num_objects":64,"num_object_clones":58,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":179,"num_read_kb":74219,"num_write":318,"num_write_kb":90184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":64}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"449'107","reported_seq":610,"reported_epoch":449,"state":"active+clean","last_fresh":"2023-03-29T14:09:56.484874+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:56.484874+0000","last_peered":"2023-03-29T14:09:56.484874+0000","last_clean":"2023-03-29T14:09:56.484874+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:56.484874+0000","last_undegraded":"2023-03-29T14:09:56.484874+0000","last_fullsized":"2023-03-29T14:09:56.484874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12908502,"num_objects":38,"num_object_clones":34,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":82092,"num_write":202,"num_write_kb":57294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":347,"reported_epoch":448,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.304571+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:09:54.304571+0000","last_peered":"2023-03-29T14:09:54.304571+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:09:54.304571+0000","last_undegraded":"2023-03-29T14:09:54.304571+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"434'111","reported_seq":380,"reported_epoch":448,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.339811+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:09:54.339811+0000","last_peered":"2023-03-29T14:09:54.339811+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:09:54.339811+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9580782,"num_objects":24,"num_object_clones":24,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":46624,"num_write":146,"num_write_kb":39178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":24}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":382,"reported_epoch":449,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.502073+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:54.502073+0000","last_peered":"2023-03-29T14:09:54.502073+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:54.502073+0000","last_undegraded":"2023-03-29T14:09:54.502073+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"434'123","reported_seq":636,"reported_epoch":448,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.322051+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:09:54.322051+0000","last_peered":"2023-03-29T14:09:54.322051+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:09:54.322051+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9614396,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":171,"num_read_kb":107609,"num_write":246,"num_write_kb":72486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":448,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.310945+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:09:54.310945+0000","last_peered":"2023-03-29T14:09:54.310945+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:09:54.310945+0000","last_undegraded":"2023-03-29T14:09:54.310945+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":715,"reported_epoch":448,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.295114+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:09:54.295114+0000","last_peered":"2023-03-29T14:09:54.295114+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:09:54.295114+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":382,"reported_epoch":449,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.505379+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:54.505379+0000","last_peered":"2023-03-29T14:09:54.505379+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:54.505379+0000","last_undegraded":"2023-03-29T14:09:54.505379+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":572,"reported_epoch":448,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.329270+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:09:54.329270+0000","last_peered":"2023-03-29T14:09:54.329270+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:09:54.329270+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"444'225","reported_seq":922,"reported_epoch":449,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.495582+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:54.495582+0000","last_peered":"2023-03-29T14:09:54.495582+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:54.495582+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14993032,"num_objects":84,"num_object_clones":76,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":394,"num_read_kb":264974,"num_write":438,"num_write_kb":126096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":84}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":382,"reported_epoch":449,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.507183+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:54.507183+0000","last_peered":"2023-03-29T14:09:54.507183+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:54.507183+0000","last_undegraded":"2023-03-29T14:09:54.507183+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"442'390","reported_seq":1230,"reported_epoch":448,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:54.316689+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:09:54.316689+0000","last_peered":"2023-03-29T14:09:54.316689+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:09:54.316689+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15161102,"num_objects":144,"num_object_clones":130,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":8,"num_read":579,"num_read_kb":355983,"num_write":722,"num_write_kb":211196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":347,"reported_epoch":448,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.313878+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:09:54.313878+0000","last_peered":"2023-03-29T14:09:54.313878+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:09:54.313878+0000","last_undegraded":"2023-03-29T14:09:54.313878+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"442'128","reported_seq":622,"reported_epoch":448,"state":"active+clean","last_fresh":"2023-03-29T14:09:54.301837+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:09:54.301837+0000","last_peered":"2023-03-29T14:09:54.301837+0000","last_clean":"2023-03-29T14:09:54.301837+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:09:54.301837+0000","last_undegraded":"2023-03-29T14:09:54.301837+0000","last_fullsized":"2023-03-29T14:09:54.301837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10421132,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":97292,"num_write":260,"num_write_kb":76088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"426'246","reported_seq":861,"reported_epoch":448,"state":"active+clean","last_fresh":"2023-03-29T14:09:54.317961+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:09:54.317961+0000","last_peered":"2023-03-29T14:09:54.317961+0000","last_clean":"2023-03-29T14:09:54.317961+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:09:54.317961+0000","last_undegraded":"2023-03-29T14:09:54.317961+0000","last_fullsized":"2023-03-29T14:09:54.317961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278190,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":306,"num_read_kb":177815,"num_write":438,"num_write_kb":126954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":347,"reported_epoch":448,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.338198+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:09:54.338198+0000","last_peered":"2023-03-29T14:09:54.338198+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:09:54.338198+0000","last_undegraded":"2023-03-29T14:09:54.338198+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":382,"reported_epoch":449,"state":"active+undersized","last_fresh":"2023-03-29T14:09:54.509567+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:54.509567+0000","last_peered":"2023-03-29T14:09:54.509567+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:54.509567+0000","last_undegraded":"2023-03-29T14:09:54.509567+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"447'256","reported_seq":928,"reported_epoch":448,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:55.338858+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:09:55.338858+0000","last_peered":"2023-03-29T14:09:55.338858+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:09:55.338858+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18791348,"num_objects":94,"num_object_clones":86,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":94,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":353,"num_read_kb":218500,"num_write":488,"num_write_kb":145666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":94}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":184553532,"num_objects":1042,"num_object_clones":950,"num_object_copies":3126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":32,"num_read":4001,"num_read_kb":2451670,"num_write":5406,"num_write_kb":1574202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2911,"ondisk_log_size":2911,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706134,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":577504,"kb_used_data":553240,"kb_used_omap":0,"kb_used_meta":24256,"kb_avail":93794336,"statfs":{"total":96636764160,"available":96045400064,"internally_reserved":0,"allocated":566517760,"data_stored":707469793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24838144},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706133,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":641428,"kb_used_data":615692,"kb_used_omap":0,"kb_used_meta":25728,"kb_avail":93730412,"statfs":{"total":96636764160,"available":95979941888,"internally_reserved":0,"allocated":630468608,"data_stored":787638063,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":26345472},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706133,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":583092,"kb_used_data":558380,"kb_used_omap":0,"kb_used_meta":24704,"kb_avail":93788748,"statfs":{"total":96636764160,"available":96039677952,"internally_reserved":0,"allocated":571781120,"data_stored":714669777,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25296896},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:10:00.953 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.3 3.0 1.0 3.2 3.6 2023-03-29T14:10:01.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:01.081+0000 7f2640263700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:01.093+0000 7f2640263700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:01.304 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.3,3.6] on osd.0 to force-backfill; instructing pg(s) [1.0,3.0] on osd.1 to force-backfill; instructing pg(s) [3.2] on osd.3 to force-backfill; 2023-03-29T14:10:01.304 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:10:01.372 INFO:tasks.rados.rados.0.smithi017.stdout:1855: done (3 left) 2023-03-29T14:10:01.372 INFO:tasks.rados.rados.0.smithi017.stdout:1856: done (2 left) 2023-03-29T14:10:01.372 INFO:tasks.rados.rados.0.smithi017.stdout:1857: done (1 left) 2023-03-29T14:10:01.372 INFO:tasks.rados.rados.0.smithi017.stdout:1859: done (0 left) 2023-03-29T14:10:01.372 INFO:tasks.rados.rados.0.smithi017.stdout:1860: write oid 33 current snap is 216 2023-03-29T14:10:01.372 INFO:tasks.rados.rados.0.smithi017.stdout:1860: seq_num 893 ranges {608651=410658,1556420=674971,2773556=235095} 2023-03-29T14:10:01.377 INFO:tasks.rados.rados.0.smithi017.stdout:1860: writing smithi017107884-33 from 608651 to 1019309 tid 1 2023-03-29T14:10:01.384 INFO:tasks.rados.rados.0.smithi017.stdout:1860: writing smithi017107884-33 from 1556420 to 2231391 tid 2 2023-03-29T14:10:01.389 INFO:tasks.rados.rados.0.smithi017.stdout:1860: writing smithi017107884-33 from 2773556 to 3008651 tid 3 2023-03-29T14:10:01.389 INFO:tasks.rados.rados.0.smithi017.stdout:1861: read oid 29 snap 215 2023-03-29T14:10:01.389 INFO:tasks.rados.rados.0.smithi017.stdout:1861: expect (ObjNum 867 snap 205 seq_num 867) 2023-03-29T14:10:01.389 INFO:tasks.rados.rados.0.smithi017.stdout:1862: snap_create 2023-03-29T14:10:01.484 INFO:tasks.rados.rados.0.smithi017.stdout:1860: finishing write tid 1 to smithi017107884-33 2023-03-29T14:10:01.499 INFO:tasks.rados.rados.0.smithi017.stdout:1860: finishing write tid 2 to smithi017107884-33 2023-03-29T14:10:01.503 INFO:tasks.rados.rados.0.smithi017.stdout:1860: finishing write tid 3 to smithi017107884-33 2023-03-29T14:10:01.512 INFO:tasks.rados.rados.0.smithi017.stdout:1860: finishing write tid 4 to smithi017107884-33 2023-03-29T14:10:01.518 INFO:tasks.rados.rados.0.smithi017.stdout:1860: finishing write tid 5 to smithi017107884-33 2023-03-29T14:10:01.518 INFO:tasks.rados.rados.0.smithi017.stdout:1860: oid 33 updating version 0 to 264 2023-03-29T14:10:01.518 INFO:tasks.rados.rados.0.smithi017.stdout:1860: oid 33 version 264 is already newer than 263 2023-03-29T14:10:01.518 INFO:tasks.rados.rados.0.smithi017.stdout:1860: oid 33 version 264 is already newer than 264 2023-03-29T14:10:01.518 INFO:tasks.rados.rados.0.smithi017.stdout:1860: oid 33 version 264 is already newer than 264 2023-03-29T14:10:01.518 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 33 v 264 (ObjNum 893 snap 216 seq_num 893) dirty exists 2023-03-29T14:10:01.518 INFO:tasks.rados.rados.0.smithi017.stdout:1860: left oid 33 (ObjNum 893 snap 216 seq_num 893) 2023-03-29T14:10:01.532 INFO:tasks.rados.rados.0.smithi017.stdout:1861: expect (ObjNum 867 snap 205 seq_num 867) 2023-03-29T14:10:01.780 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~292s 2023-03-29T14:10:02.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:10:02.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:02.192+0000 7fb3ca948700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:02.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:02.196+0000 7fb3ca948700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:02.259 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:10:02.262 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:10:02.262 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:10:02.380 INFO:tasks.rados.rados.0.smithi017.stdout:1860: done (2 left) 2023-03-29T14:10:02.380 INFO:tasks.rados.rados.0.smithi017.stdout:1861: done (1 left) 2023-03-29T14:10:02.380 INFO:tasks.rados.rados.0.smithi017.stdout:1862: done (0 left) 2023-03-29T14:10:02.380 INFO:tasks.rados.rados.0.smithi017.stdout:1863: snap_remove snap 211 2023-03-29T14:10:03.269 INFO:tasks.rados.rados.0.smithi017.stdout:1863: done (0 left) 2023-03-29T14:10:03.269 INFO:tasks.rados.rados.0.smithi017.stdout:1864: read oid 7 snap -1 2023-03-29T14:10:03.270 INFO:tasks.rados.rados.0.smithi017.stdout:1864: expect (ObjNum 807 snap 196 seq_num 807) 2023-03-29T14:10:03.270 INFO:tasks.rados.rados.0.smithi017.stdout:1865: write (excl) oid 13 current snap is 217 2023-03-29T14:10:03.270 INFO:tasks.rados.rados.0.smithi017.stdout:1865: seq_num 894 ranges {625458=485907,1614902=534982,2917332=108126} 2023-03-29T14:10:03.277 INFO:tasks.rados.rados.0.smithi017.stdout:1865: writing smithi017107884-13 from 625458 to 1111365 tid 1 2023-03-29T14:10:03.286 INFO:tasks.rados.rados.0.smithi017.stdout:1865: writing smithi017107884-13 from 1614902 to 2149884 tid 2 2023-03-29T14:10:03.292 INFO:tasks.rados.rados.0.smithi017.stdout:1865: writing smithi017107884-13 from 2917332 to 3025458 tid 3 2023-03-29T14:10:03.292 INFO:tasks.rados.rados.0.smithi017.stdout:1866: write (excl) oid 25 current snap is 217 2023-03-29T14:10:03.292 INFO:tasks.rados.rados.0.smithi017.stdout:1866: seq_num 895 ranges {642265=561156,1673384=678640,2828402=213863} 2023-03-29T14:10:03.301 INFO:tasks.rados.rados.0.smithi017.stdout:1866: writing smithi017107884-25 from 642265 to 1203421 tid 1 2023-03-29T14:10:03.310 INFO:tasks.rados.rados.0.smithi017.stdout:1866: writing smithi017107884-25 from 1673384 to 2352024 tid 2 2023-03-29T14:10:03.316 INFO:tasks.rados.rados.0.smithi017.stdout:1866: writing smithi017107884-25 from 2828402 to 3042265 tid 3 2023-03-29T14:10:03.316 INFO:tasks.rados.rados.0.smithi017.stdout:1867: read oid 22 snap -1 2023-03-29T14:10:03.316 INFO:tasks.rados.rados.0.smithi017.stdout:1867: expect (ObjNum 856 snap 204 seq_num 856) 2023-03-29T14:10:03.316 INFO:tasks.rados.rados.0.smithi017.stdout:1868: read oid 45 snap 215 2023-03-29T14:10:03.317 INFO:tasks.rados.rados.0.smithi017.stdout:1868: expect deleted 2023-03-29T14:10:03.317 INFO:tasks.rados.rados.0.smithi017.stdout:1869: delete oid 37 current snap is 217 2023-03-29T14:10:03.317 INFO:tasks.rados.rados.0.smithi017.stdout:1864: expect (ObjNum 807 snap 196 seq_num 807) 2023-03-29T14:10:03.503 INFO:tasks.rados.rados.0.smithi017.stdout:1867: expect (ObjNum 856 snap 204 seq_num 856) 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1866: finishing write tid 1 to smithi017107884-25 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1865: finishing write tid 1 to smithi017107884-13 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1866: finishing write tid 2 to smithi017107884-25 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1866: finishing write tid 3 to smithi017107884-25 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1866: finishing write tid 4 to smithi017107884-25 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1864: done (5 left) 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1867: done (4 left) 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1868: done (3 left) 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1869: done (2 left) 2023-03-29T14:10:03.717 INFO:tasks.rados.rados.0.smithi017.stdout:1870: snap_create 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: finishing write tid 2 to smithi017107884-13 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1866: finishing write tid 5 to smithi017107884-25 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1866: oid 25 updating version 0 to 296 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1866: oid 25 updating version 296 to 299 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1866: oid 25 version 299 is already newer than 299 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1866: oid 25 version 299 is already newer than 299 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 25 v 299 (ObjNum 895 snap 217 seq_num 895) dirty exists 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1866: left oid 25 (ObjNum 895 snap 217 seq_num 895) 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: finishing write tid 3 to smithi017107884-13 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: finishing write tid 4 to smithi017107884-13 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: finishing write tid 5 to smithi017107884-13 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: oid 13 updating version 0 to 254 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: oid 13 version 254 is already newer than 254 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: oid 13 version 254 is already newer than 254 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: oid 13 version 254 is already newer than 251 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 13 v 254 (ObjNum 894 snap 217 seq_num 894) dirty exists 2023-03-29T14:10:03.718 INFO:tasks.rados.rados.0.smithi017.stdout:1865: left oid 13 (ObjNum 894 snap 217 seq_num 894) 2023-03-29T14:10:04.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:10:04.385 INFO:tasks.rados.rados.0.smithi017.stdout:1865: done (2 left) 2023-03-29T14:10:04.385 INFO:tasks.rados.rados.0.smithi017.stdout:1866: done (1 left) 2023-03-29T14:10:04.385 INFO:tasks.rados.rados.0.smithi017.stdout:1870: done (0 left) 2023-03-29T14:10:04.385 INFO:tasks.rados.rados.0.smithi017.stdout:1871: read oid 2 snap -1 2023-03-29T14:10:04.385 INFO:tasks.rados.rados.0.smithi017.stdout:1871: expect deleted 2023-03-29T14:10:04.385 INFO:tasks.rados.rados.0.smithi017.stdout:1872: write (excl) oid 40 current snap is 218 2023-03-29T14:10:04.385 INFO:tasks.rados.rados.0.smithi017.stdout:1872: seq_num 897 ranges {675879=711654,2073995=682309} 2023-03-29T14:10:04.393 INFO:tasks.rados.rados.0.smithi017.stdout:1872: writing smithi017107884-40 from 675879 to 1387533 tid 1 2023-03-29T14:10:04.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:04.394+0000 7f4c17378700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:04.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:04.399+0000 7f4c17378700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:04.402 INFO:tasks.rados.rados.0.smithi017.stdout:1872: writing smithi017107884-40 from 2073995 to 2756304 tid 2 2023-03-29T14:10:04.402 INFO:tasks.rados.rados.0.smithi017.stdout:1873: read oid 29 snap -1 2023-03-29T14:10:04.402 INFO:tasks.rados.rados.0.smithi017.stdout:1873: expect (ObjNum 867 snap 205 seq_num 867) 2023-03-29T14:10:04.402 INFO:tasks.rados.rados.0.smithi017.stdout:1874: write oid 5 current snap is 218 2023-03-29T14:10:04.403 INFO:tasks.rados.rados.0.smithi017.stdout:1874: seq_num 898 ranges {692686=503256,1848830=425967,2910671=182015} 2023-03-29T14:10:04.408 INFO:tasks.rados.rados.0.smithi017.stdout:1874: writing smithi017107884-5 from 692686 to 1195942 tid 1 2023-03-29T14:10:04.414 INFO:tasks.rados.rados.0.smithi017.stdout:1874: writing smithi017107884-5 from 1848830 to 2274797 tid 2 2023-03-29T14:10:04.419 INFO:tasks.rados.rados.0.smithi017.stdout:1874: writing smithi017107884-5 from 2910671 to 3092686 tid 3 2023-03-29T14:10:04.419 INFO:tasks.rados.rados.0.smithi017.stdout:1875: write (excl) oid 49 current snap is 218 2023-03-29T14:10:04.419 INFO:tasks.rados.rados.0.smithi017.stdout:1875: seq_num 899 ranges {709493=578505,1907312=685978} 2023-03-29T14:10:04.426 INFO:tasks.rados.rados.0.smithi017.stdout:1875: writing smithi017107884-49 from 709493 to 1287998 tid 1 2023-03-29T14:10:04.434 INFO:tasks.rados.rados.0.smithi017.stdout:1875: writing smithi017107884-49 from 1907312 to 2593290 tid 2 2023-03-29T14:10:04.434 INFO:tasks.rados.rados.0.smithi017.stdout:1876: read oid 45 snap 217 2023-03-29T14:10:04.434 INFO:tasks.rados.rados.0.smithi017.stdout:1876: expect deleted 2023-03-29T14:10:04.434 INFO:tasks.rados.rados.0.smithi017.stdout:1877: snap_create 2023-03-29T14:10:04.539 INFO:tasks.rados.rados.0.smithi017.stdout:1872: finishing write tid 1 to smithi017107884-40 2023-03-29T14:10:04.547 INFO:tasks.rados.rados.0.smithi017.stdout:1873: expect (ObjNum 867 snap 205 seq_num 867) 2023-03-29T14:10:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:10:04.623 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:10:04.623 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:10:04.715 INFO:tasks.rados.rados.0.smithi017.stdout:1874: finishing write tid 1 to smithi017107884-5 2023-03-29T14:10:04.715 INFO:tasks.rados.rados.0.smithi017.stdout:1874: finishing write tid 2 to smithi017107884-5 2023-03-29T14:10:04.715 INFO:tasks.rados.rados.0.smithi017.stdout:1875: finishing write tid 1 to smithi017107884-49 2023-03-29T14:10:04.715 INFO:tasks.rados.rados.0.smithi017.stdout:1872: finishing write tid 2 to smithi017107884-40 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1872: finishing write tid 3 to smithi017107884-40 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1875: finishing write tid 2 to smithi017107884-49 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1872: finishing write tid 4 to smithi017107884-40 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1872: oid 40 updating version 0 to 303 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1872: oid 40 version 303 is already newer than 300 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1872: oid 40 version 303 is already newer than 303 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 40 v 303 (ObjNum 897 snap 218 seq_num 897) dirty exists 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1872: left oid 40 (ObjNum 897 snap 218 seq_num 897) 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: finishing write tid 3 to smithi017107884-5 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: finishing write tid 4 to smithi017107884-5 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: finishing write tid 5 to smithi017107884-5 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: oid 5 updating version 0 to 258 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: oid 5 version 258 is already newer than 258 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: oid 5 version 258 is already newer than 258 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: oid 5 version 258 is already newer than 258 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 258 (ObjNum 898 snap 218 seq_num 898) dirty exists 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1874: left oid 5 (ObjNum 898 snap 218 seq_num 898) 2023-03-29T14:10:04.716 INFO:tasks.rados.rados.0.smithi017.stdout:1875: finishing write tid 3 to smithi017107884-49 2023-03-29T14:10:04.717 INFO:tasks.rados.rados.0.smithi017.stdout:1875: finishing write tid 4 to smithi017107884-49 2023-03-29T14:10:04.717 INFO:tasks.rados.rados.0.smithi017.stdout:1875: oid 49 updating version 0 to 398 2023-03-29T14:10:04.717 INFO:tasks.rados.rados.0.smithi017.stdout:1875: oid 49 version 398 is already newer than 398 2023-03-29T14:10:04.717 INFO:tasks.rados.rados.0.smithi017.stdout:1875: oid 49 version 398 is already newer than 395 2023-03-29T14:10:04.717 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 49 v 398 (ObjNum 899 snap 218 seq_num 899) dirty exists 2023-03-29T14:10:04.717 INFO:tasks.rados.rados.0.smithi017.stdout:1875: left oid 49 (ObjNum 899 snap 218 seq_num 899) 2023-03-29T14:10:05.387 INFO:tasks.rados.rados.0.smithi017.stdout:1871: done (6 left) 2023-03-29T14:10:05.387 INFO:tasks.rados.rados.0.smithi017.stdout:1872: done (5 left) 2023-03-29T14:10:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:1873: done (4 left) 2023-03-29T14:10:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:1874: done (3 left) 2023-03-29T14:10:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:1875: done (2 left) 2023-03-29T14:10:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:1876: done (1 left) 2023-03-29T14:10:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:1877: done (0 left) 2023-03-29T14:10:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:1878: write (excl) oid 16 current snap is 219 2023-03-29T14:10:05.388 INFO:tasks.rados.rados.0.smithi017.stdout:1878: seq_num 900 ranges {726300=653754,1965794=429636,2965517=160783} 2023-03-29T14:10:05.395 INFO:tasks.rados.rados.0.smithi017.stdout:1878: writing smithi017107884-16 from 726300 to 1380054 tid 1 2023-03-29T14:10:05.400 INFO:tasks.rados.rados.0.smithi017.stdout:1878: writing smithi017107884-16 from 1965794 to 2395430 tid 2 2023-03-29T14:10:05.404 INFO:tasks.rados.rados.0.smithi017.stdout:1878: writing smithi017107884-16 from 2965517 to 3126300 tid 3 2023-03-29T14:10:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:1879: read oid 44 snap -1 2023-03-29T14:10:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:1879: expect (ObjNum 861 snap 205 seq_num 861) 2023-03-29T14:10:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:1880: read oid 29 snap -1 2023-03-29T14:10:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:1880: expect (ObjNum 867 snap 205 seq_num 867) 2023-03-29T14:10:05.405 INFO:tasks.rados.rados.0.smithi017.stdout:1881: snap_create 2023-03-29T14:10:05.483 INFO:tasks.rados.rados.0.smithi017.stdout:1878: finishing write tid 1 to smithi017107884-16 2023-03-29T14:10:05.510 INFO:tasks.rados.rados.0.smithi017.stdout:1879: expect (ObjNum 861 snap 205 seq_num 861) 2023-03-29T14:10:05.604 INFO:tasks.rados.rados.0.smithi017.stdout:1880: expect (ObjNum 867 snap 205 seq_num 867) 2023-03-29T14:10:05.768 INFO:tasks.rados.rados.0.smithi017.stdout:1878: finishing write tid 2 to smithi017107884-16 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: finishing write tid 3 to smithi017107884-16 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: finishing write tid 4 to smithi017107884-16 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: finishing write tid 5 to smithi017107884-16 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: oid 16 updating version 0 to 138 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: oid 16 version 138 is already newer than 138 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: oid 16 version 138 is already newer than 138 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: oid 16 version 138 is already newer than 134 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 16 v 138 (ObjNum 900 snap 219 seq_num 900) dirty exists 2023-03-29T14:10:05.769 INFO:tasks.rados.rados.0.smithi017.stdout:1878: left oid 16 (ObjNum 900 snap 219 seq_num 900) 2023-03-29T14:10:06.316 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:10:06.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:10:06.317 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:10:06.391 INFO:tasks.rados.rados.0.smithi017.stdout:1878: done (3 left) 2023-03-29T14:10:06.391 INFO:tasks.rados.rados.0.smithi017.stdout:1879: done (2 left) 2023-03-29T14:10:06.391 INFO:tasks.rados.rados.0.smithi017.stdout:1880: done (1 left) 2023-03-29T14:10:06.391 INFO:tasks.rados.rados.0.smithi017.stdout:1881: done (0 left) 2023-03-29T14:10:06.391 INFO:tasks.rados.rados.0.smithi017.stdout:1882: write oid 5 current snap is 220 2023-03-29T14:10:06.391 INFO:tasks.rados.rados.0.smithi017.stdout:1882: seq_num 901 ranges {743107=729003,1907923=689647} 2023-03-29T14:10:06.399 INFO:tasks.rados.rados.0.smithi017.stdout:1882: writing smithi017107884-5 from 743107 to 1472110 tid 1 2023-03-29T14:10:06.406 INFO:tasks.rados.rados.0.smithi017.stdout:1882: writing smithi017107884-5 from 1907923 to 2597570 tid 2 2023-03-29T14:10:06.406 INFO:tasks.rados.rados.0.smithi017.stdout:1883: read oid 14 snap -1 2023-03-29T14:10:06.406 INFO:tasks.rados.rados.0.smithi017.stdout:1883: expect (ObjNum 887 snap 214 seq_num 887) 2023-03-29T14:10:06.406 INFO:tasks.rados.rados.0.smithi017.stdout:1884: write oid 42 current snap is 220 2023-03-29T14:10:06.406 INFO:tasks.rados.rados.0.smithi017.stdout:1884: seq_num 902 ranges {759914=404252,1566405=433305,2504010=655904} 2023-03-29T14:10:06.412 INFO:tasks.rados.rados.0.smithi017.stdout:1884: writing smithi017107884-42 from 759914 to 1164166 tid 1 2023-03-29T14:10:06.417 INFO:tasks.rados.rados.0.smithi017.stdout:1884: writing smithi017107884-42 from 1566405 to 1999710 tid 2 2023-03-29T14:10:06.423 INFO:tasks.rados.rados.0.smithi017.stdout:1884: writing smithi017107884-42 from 2504010 to 3159914 tid 3 2023-03-29T14:10:06.424 INFO:tasks.rados.rados.0.smithi017.stdout:1885: read oid 7 snap -1 2023-03-29T14:10:06.424 INFO:tasks.rados.rados.0.smithi017.stdout:1885: expect (ObjNum 807 snap 196 seq_num 807) 2023-03-29T14:10:06.424 INFO:tasks.rados.rados.0.smithi017.stdout:1886: read oid 17 snap -1 2023-03-29T14:10:06.424 INFO:tasks.rados.rados.0.smithi017.stdout:1886: expect (ObjNum 764 snap 188 seq_num 764) 2023-03-29T14:10:06.424 INFO:tasks.rados.rados.0.smithi017.stdout:1887: read oid 45 snap -1 2023-03-29T14:10:06.424 INFO:tasks.rados.rados.0.smithi017.stdout:1887: expect deleted 2023-03-29T14:10:06.424 INFO:tasks.rados.rados.0.smithi017.stdout:1888: snap_create 2023-03-29T14:10:06.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:06.447+0000 7fc4f9632700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:06.454 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:06.454+0000 7fc4f9632700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:06.468 INFO:tasks.rados.rados.0.smithi017.stdout:1883: expect (ObjNum 887 snap 214 seq_num 887) 2023-03-29T14:10:06.625 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:10:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:10:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:10:06.688 INFO:tasks.rados.rados.0.smithi017.stdout:1885: expect (ObjNum 807 snap 196 seq_num 807) 2023-03-29T14:10:06.693 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":693,"stamp":"2023-03-29T14:10:06.389484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197432336,"num_objects":1056,"num_object_clones":960,"num_object_copies":3168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":4226,"num_read_kb":2509586,"num_write":5738,"num_write_kb":1592918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3045,"ondisk_log_size":3045,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":1986740,"kb_used_data":1899348,"kb_used_omap":0,"kb_used_meta":87360,"kb_avail":375500620,"statfs":{"total":386547056640,"available":384512634880,"internally_reserved":0,"allocated":1944932352,"data_stored":2419072316,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":89456640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9983754,"num_objects":6,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":38,"num_read_kb":25833,"num_write":38,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012266"},"pg_stats":[{"pgid":"3.f","version":"432'133","reported_seq":447,"reported_epoch":454,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:58.391001+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:09:58.391001+0000","last_peered":"2023-03-29T14:09:58.391001+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:09:58.391001+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202608,"num_objects":30,"num_object_clones":30,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65947,"num_write":180,"num_write_kb":53646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":30}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":479,"reported_epoch":458,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:04.468366+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:10:04.468366+0000","last_peered":"2023-03-29T14:10:04.468366+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:10:04.468366+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":39146,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"451'271","reported_seq":990,"reported_epoch":454,"state":"active+clean","last_fresh":"2023-03-29T14:09:58.791210+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:09:58.791210+0000","last_peered":"2023-03-29T14:09:58.791210+0000","last_clean":"2023-03-29T14:09:58.791210+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:09:58.791210+0000","last_undegraded":"2023-03-29T14:09:58.791210+0000","last_fullsized":"2023-03-29T14:09:58.791210+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11765560,"num_objects":102,"num_object_clones":92,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":419,"num_read_kb":296167,"num_write":512,"num_write_kb":152358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"437'295","reported_seq":1028,"reported_epoch":454,"state":"active+clean","last_fresh":"2023-03-29T14:09:58.381786+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:09:58.381786+0000","last_peered":"2023-03-29T14:09:58.381786+0000","last_clean":"2023-03-29T14:09:58.381786+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:09:58.381786+0000","last_undegraded":"2023-03-29T14:09:58.381786+0000","last_fullsized":"2023-03-29T14:09:58.381786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15127422,"num_objects":118,"num_object_clones":110,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":411,"num_read_kb":243746,"num_write":578,"num_write_kb":162888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"451'233","reported_seq":891,"reported_epoch":458,"state":"active+clean","last_fresh":"2023-03-29T14:10:04.458292+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:10:04.458292+0000","last_peered":"2023-03-29T14:10:04.458292+0000","last_clean":"2023-03-29T14:10:04.458292+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:10:04.458292+0000","last_undegraded":"2023-03-29T14:10:04.458292+0000","last_fullsized":"2023-03-29T14:10:04.458292+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10320092,"num_objects":84,"num_object_clones":76,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":368,"num_read_kb":246693,"num_write":460,"num_write_kb":135804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":381,"reported_epoch":454,"state":"active+undersized","last_fresh":"2023-03-29T14:09:58.382326+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:09:58.382326+0000","last_peered":"2023-03-29T14:09:58.382326+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:09:58.382326+0000","last_undegraded":"2023-03-29T14:09:58.382326+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"449'178","reported_seq":739,"reported_epoch":454,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:58.393633+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:09:58.393633+0000","last_peered":"2023-03-29T14:09:58.393633+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:09:58.393633+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5815618,"num_objects":64,"num_object_clones":58,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":180,"num_read_kb":74220,"num_write":318,"num_write_kb":90184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":64}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"449'108","reported_seq":613,"reported_epoch":454,"state":"active+clean","last_fresh":"2023-03-29T14:09:58.394402+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:09:58.394402+0000","last_peered":"2023-03-29T14:09:58.394402+0000","last_clean":"2023-03-29T14:09:58.394402+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:09:58.394402+0000","last_undegraded":"2023-03-29T14:09:58.394402+0000","last_fullsized":"2023-03-29T14:09:58.394402+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12908502,"num_objects":38,"num_object_clones":34,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":82093,"num_write":202,"num_write_kb":57294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":355,"reported_epoch":458,"state":"active+undersized","last_fresh":"2023-03-29T14:10:04.473110+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:10:04.473110+0000","last_peered":"2023-03-29T14:10:04.473110+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:10:04.473110+0000","last_undegraded":"2023-03-29T14:10:04.473110+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"456'113","reported_seq":394,"reported_epoch":458,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:04.433042+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:10:04.433042+0000","last_peered":"2023-03-29T14:10:04.433042+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:10:04.433042+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4672742,"num_objects":26,"num_object_clones":26,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":97,"num_read_kb":52640,"num_write":146,"num_write_kb":39178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":384,"reported_epoch":454,"state":"active+undersized","last_fresh":"2023-03-29T14:09:58.389254+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:09:58.389254+0000","last_peered":"2023-03-29T14:09:58.389254+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:09:58.389254+0000","last_undegraded":"2023-03-29T14:09:58.389254+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"434'123","reported_seq":657,"reported_epoch":458,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:05.496450+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:10:05.496450+0000","last_peered":"2023-03-29T14:10:05.496450+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:10:05.496450+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9614396,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":199,"num_read_kb":133887,"num_write":246,"num_write_kb":72486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":458,"state":"active+undersized","last_fresh":"2023-03-29T14:10:04.459423+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:10:04.459423+0000","last_peered":"2023-03-29T14:10:04.459423+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:10:04.459423+0000","last_undegraded":"2023-03-29T14:10:04.459423+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":723,"reported_epoch":458,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:04.466378+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:10:04.466378+0000","last_peered":"2023-03-29T14:10:04.466378+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:10:04.466378+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":385,"reported_epoch":454,"state":"active+undersized","last_fresh":"2023-03-29T14:09:59.394342+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:09:59.394342+0000","last_peered":"2023-03-29T14:09:59.394342+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:09:59.394342+0000","last_undegraded":"2023-03-29T14:09:59.394342+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":580,"reported_epoch":458,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:04.494409+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:10:04.494409+0000","last_peered":"2023-03-29T14:10:04.494409+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:10:04.494409+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"444'225","reported_seq":925,"reported_epoch":454,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:09:59.403807+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:09:59.403807+0000","last_peered":"2023-03-29T14:09:59.403807+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:09:59.403807+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14993032,"num_objects":84,"num_object_clones":76,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":394,"num_read_kb":264974,"num_write":438,"num_write_kb":126096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":84}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":384,"reported_epoch":454,"state":"active+undersized","last_fresh":"2023-03-29T14:09:58.396354+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:09:58.396354+0000","last_peered":"2023-03-29T14:09:58.396354+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:09:58.396354+0000","last_undegraded":"2023-03-29T14:09:58.396354+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"457'398","reported_seq":1255,"reported_epoch":458,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:04.693373+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:10:04.693373+0000","last_peered":"2023-03-29T14:10:04.693373+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:10:04.693373+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22926332,"num_objects":146,"num_object_clones":132,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":585,"num_read_kb":355985,"num_write":740,"num_write_kb":216430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":355,"reported_epoch":458,"state":"active+undersized","last_fresh":"2023-03-29T14:10:04.430189+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:10:04.430189+0000","last_peered":"2023-03-29T14:10:04.430189+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:10:04.430189+0000","last_undegraded":"2023-03-29T14:10:04.430189+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"442'128","reported_seq":633,"reported_epoch":458,"state":"active+clean","last_fresh":"2023-03-29T14:10:05.503630+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:10:05.503630+0000","last_peered":"2023-03-29T14:10:05.503630+0000","last_clean":"2023-03-29T14:10:05.503630+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:10:05.503630+0000","last_undegraded":"2023-03-29T14:10:05.503630+0000","last_fullsized":"2023-03-29T14:10:05.503630+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10421132,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":102553,"num_write":260,"num_write_kb":76088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"457'258","reported_seq":896,"reported_epoch":458,"state":"active+clean","last_fresh":"2023-03-29T14:10:04.670823+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:10:04.670823+0000","last_peered":"2023-03-29T14:10:04.670823+0000","last_clean":"2023-03-29T14:10:04.670823+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:10:04.670823+0000","last_undegraded":"2023-03-29T14:10:04.670823+0000","last_fullsized":"2023-03-29T14:10:04.670823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27498166,"num_objects":88,"num_object_clones":78,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":177818,"num_write":468,"num_write_kb":134064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":355,"reported_epoch":458,"state":"active+undersized","last_fresh":"2023-03-29T14:10:04.445476+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:10:04.445476+0000","last_peered":"2023-03-29T14:10:04.445476+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:10:04.445476+0000","last_undegraded":"2023-03-29T14:10:04.445476+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":384,"reported_epoch":454,"state":"active+undersized","last_fresh":"2023-03-29T14:09:58.388504+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:09:58.388504+0000","last_peered":"2023-03-29T14:09:58.388504+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:09:58.388504+0000","last_undegraded":"2023-03-29T14:09:58.388504+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"454'264","reported_seq":947,"reported_epoch":458,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:04.443663+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:10:04.443663+0000","last_peered":"2023-03-29T14:10:04.443663+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:10:04.443663+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19530856,"num_objects":96,"num_object_clones":88,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":96,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":354,"num_read_kb":218501,"num_write":498,"num_write_kb":148250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":96}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":196251600,"num_objects":1052,"num_object_clones":960,"num_object_copies":3156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":576,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":30,"num_read":4084,"num_read_kb":2509466,"num_write":5464,"num_write_kb":1589130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2948,"ondisk_log_size":2948,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706139,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":582860,"kb_used_data":558468,"kb_used_omap":0,"kb_used_meta":24384,"kb_avail":93788980,"statfs":{"total":96636764160,"available":96039915520,"internally_reserved":0,"allocated":571871232,"data_stored":714105143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":24969216},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":654456,"kb_used_data":628464,"kb_used_omap":0,"kb_used_meta":25984,"kb_avail":93717384,"statfs":{"total":96636764160,"available":95966601216,"internally_reserved":0,"allocated":643547136,"data_stored":804087769,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":26607616},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":593480,"kb_used_data":568576,"kb_used_omap":0,"kb_used_meta":24896,"kb_avail":93778360,"statfs":{"total":96636764160,"available":96029040640,"internally_reserved":0,"allocated":582221824,"data_stored":727647179,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25493504},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:10:06.697 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.9 3.3 3.0 3.2 3.6 2023-03-29T14:10:06.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:06.761+0000 7f793337a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:06.766 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:06.765+0000 7f793337a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:06.839 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:10:06.843 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:10:06.843 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:10:06.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:06.830+0000 7fa6b8592700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:06.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:06.850+0000 7fa6b8592700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:06.884 INFO:tasks.rados.rados.0.smithi017.stdout:1886: expect (ObjNum 764 snap 188 seq_num 764) 2023-03-29T14:10:07.057 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.3,3.6] on osd.0 to force-recovery; instructing pg(s) [3.0] on osd.1 to force-recovery; instructing pg(s) [3.2,3.9] on osd.3 to force-recovery; 2023-03-29T14:10:07.057 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1884: finishing write tid 1 to smithi017107884-42 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1884: finishing write tid 2 to smithi017107884-42 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1884: finishing write tid 3 to smithi017107884-42 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: finishing write tid 1 to smithi017107884-5 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: finishing write tid 2 to smithi017107884-5 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: finishing write tid 3 to smithi017107884-5 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: finishing write tid 4 to smithi017107884-5 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: oid 5 updating version 0 to 262 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: oid 5 version 262 is already newer than 262 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: oid 5 version 262 is already newer than 262 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 5 v 262 (ObjNum 901 snap 220 seq_num 901) dirty exists 2023-03-29T14:10:07.073 INFO:tasks.rados.rados.0.smithi017.stdout:1882: left oid 5 (ObjNum 901 snap 220 seq_num 901) 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:1884: finishing write tid 4 to smithi017107884-42 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:1884: finishing write tid 5 to smithi017107884-42 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:1884: oid 42 updating version 0 to 308 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:1884: oid 42 version 308 is already newer than 308 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:1884: oid 42 version 308 is already newer than 308 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:1884: oid 42 version 308 is already newer than 308 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 42 v 308 (ObjNum 902 snap 220 seq_num 902) dirty exists 2023-03-29T14:10:07.074 INFO:tasks.rados.rados.0.smithi017.stdout:1884: left oid 42 (ObjNum 902 snap 220 seq_num 902) 2023-03-29T14:10:07.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~297s 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1882: done (6 left) 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1883: done (5 left) 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1884: done (4 left) 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1885: done (3 left) 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1886: done (2 left) 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1887: done (1 left) 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1888: done (0 left) 2023-03-29T14:10:07.395 INFO:tasks.rados.rados.0.smithi017.stdout:1889: snap_remove snap 218 2023-03-29T14:10:08.398 INFO:tasks.rados.rados.0.smithi017.stdout:1889: done (0 left) 2023-03-29T14:10:08.398 INFO:tasks.rados.rados.0.smithi017.stdout:1890: write (excl) oid 19 current snap is 221 2023-03-29T14:10:08.398 INFO:tasks.rados.rados.0.smithi017.stdout:1890: seq_num 903 ranges {776721=479501,2024887=693316} 2023-03-29T14:10:08.405 INFO:tasks.rados.rados.0.smithi017.stdout:1890: writing smithi017107884-19 from 776721 to 1256222 tid 1 2023-03-29T14:10:08.413 INFO:tasks.rados.rados.0.smithi017.stdout:1890: writing smithi017107884-19 from 2024887 to 2718203 tid 2 2023-03-29T14:10:08.413 INFO:tasks.rados.rados.0.smithi017.stdout:1891: read oid 50 snap 217 2023-03-29T14:10:08.413 INFO:tasks.rados.rados.0.smithi017.stdout:1891: expect deleted 2023-03-29T14:10:08.413 INFO:tasks.rados.rados.0.smithi017.stdout:1892: delete oid 6 current snap is 221 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1891: done (2 left) 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1892: done (1 left) 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1893: read oid 6 snap -1 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1893: expect deleted 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1894: read oid 5 snap 215 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1894: expect deleted 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1895: read oid 8 snap -1 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1895: expect deleted 2023-03-29T14:10:08.571 INFO:tasks.rados.rados.0.smithi017.stdout:1896: snap_remove snap 220 2023-03-29T14:10:08.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:10:08.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:08.972+0000 7f04a10f9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:08.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:08.976+0000 7f04a10f9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:09.038 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:10:09.040 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:10:09.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:10:09.402 INFO:tasks.rados.rados.0.smithi017.stdout:1896: done (4 left) 2023-03-29T14:10:09.402 INFO:tasks.rados.rados.0.smithi017.stdout:1897: snap_remove snap 214 2023-03-29T14:10:10.405 INFO:tasks.rados.rados.0.smithi017.stdout:1897: done (4 left) 2023-03-29T14:10:10.405 INFO:tasks.rados.rados.0.smithi017.stdout:1898: write oid 18 current snap is 221 2023-03-29T14:10:10.405 INFO:tasks.rados.rados.0.smithi017.stdout:1898: seq_num 905 ranges {410335=746352,1741851=696985,3102632=107703} 2023-03-29T14:10:10.411 INFO:tasks.rados.rados.0.smithi017.stdout:1898: writing smithi017107884-18 from 410335 to 1156687 tid 1 2023-03-29T14:10:10.418 INFO:tasks.rados.rados.0.smithi017.stdout:1898: writing smithi017107884-18 from 1741851 to 2438836 tid 2 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1898: writing smithi017107884-18 from 3102632 to 3210335 tid 3 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1899: read oid 33 snap -1 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1899: expect (ObjNum 893 snap 216 seq_num 893) 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1900: write oid 1 current snap is 221 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: finishing write tid 1 to smithi017107884-19 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: finishing write tid 2 to smithi017107884-19 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: finishing write tid 3 to smithi017107884-19 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: finishing write tid 4 to smithi017107884-19 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: oid 19 updating version 0 to 268 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: oid 19 version 268 is already newer than 268 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: oid 19 version 268 is already newer than 265 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 19 v 268 (ObjNum 903 snap 221 seq_num 903) dirty exists 2023-03-29T14:10:10.423 INFO:tasks.rados.rados.0.smithi017.stdout:1890: left oid 19 (ObjNum 903 snap 221 seq_num 903) 2023-03-29T14:10:10.424 INFO:tasks.rados.rados.0.smithi017.stdout:1900: seq_num 906 ranges {427142=421601,1400333=440643,2613702=450326} 2023-03-29T14:10:10.427 INFO:tasks.rados.rados.0.smithi017.stdout:1900: writing smithi017107884-1 from 427142 to 848743 tid 1 2023-03-29T14:10:10.433 INFO:tasks.rados.rados.0.smithi017.stdout:1900: writing smithi017107884-1 from 1400333 to 1840976 tid 2 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1900: writing smithi017107884-1 from 2613702 to 3064028 tid 3 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1890: done (6 left) 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1893: done (5 left) 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1894: done (4 left) 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1901: read oid 13 snap -1 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1901: expect (ObjNum 894 snap 217 seq_num 894) 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1902: write (excl) oid 27 current snap is 221 2023-03-29T14:10:10.440 INFO:tasks.rados.rados.0.smithi017.stdout:1902: seq_num 907 ranges {443949=496850,1458815=584301,2641125=602824} 2023-03-29T14:10:10.445 INFO:tasks.rados.rados.0.smithi017.stdout:1902: writing smithi017107884-27 from 443949 to 940799 tid 1 2023-03-29T14:10:10.453 INFO:tasks.rados.rados.0.smithi017.stdout:1902: writing smithi017107884-27 from 1458815 to 2043116 tid 2 2023-03-29T14:10:10.460 INFO:tasks.rados.rados.0.smithi017.stdout:1902: writing smithi017107884-27 from 2641125 to 3243949 tid 3 2023-03-29T14:10:10.460 INFO:tasks.rados.rados.0.smithi017.stdout:1895: done (5 left) 2023-03-29T14:10:10.460 INFO:tasks.rados.rados.0.smithi017.stdout:1903: write (excl) oid 46 current snap is 221 2023-03-29T14:10:10.460 INFO:tasks.rados.rados.0.smithi017.stdout:1898: finishing write tid 1 to smithi017107884-18 2023-03-29T14:10:10.461 INFO:tasks.rados.rados.0.smithi017.stdout:1903: seq_num 908 ranges {460756=572099,1517297=444312,2668548=472870} 2023-03-29T14:10:10.467 INFO:tasks.rados.rados.0.smithi017.stdout:1903: writing smithi017107884-46 from 460756 to 1032855 tid 1 2023-03-29T14:10:10.472 INFO:tasks.rados.rados.0.smithi017.stdout:1903: writing smithi017107884-46 from 1517297 to 1961609 tid 2 2023-03-29T14:10:10.479 INFO:tasks.rados.rados.0.smithi017.stdout:1903: writing smithi017107884-46 from 2668548 to 3141418 tid 3 2023-03-29T14:10:10.479 INFO:tasks.rados.rados.0.smithi017.stdout:1904: snap_create 2023-03-29T14:10:10.499 INFO:tasks.rados.rados.0.smithi017.stdout:1898: finishing write tid 2 to smithi017107884-18 2023-03-29T14:10:10.535 INFO:tasks.rados.rados.0.smithi017.stdout:1899: expect (ObjNum 893 snap 216 seq_num 893) 2023-03-29T14:10:10.693 INFO:tasks.rados.rados.0.smithi017.stdout:1898: finishing write tid 3 to smithi017107884-18 2023-03-29T14:10:10.693 INFO:tasks.rados.rados.0.smithi017.stdout:1898: finishing write tid 4 to smithi017107884-18 2023-03-29T14:10:10.693 INFO:tasks.rados.rados.0.smithi017.stdout:1898: finishing write tid 5 to smithi017107884-18 2023-03-29T14:10:10.693 INFO:tasks.rados.rados.0.smithi017.stdout:1898: oid 18 updating version 0 to 182 2023-03-29T14:10:10.693 INFO:tasks.rados.rados.0.smithi017.stdout:1898: oid 18 version 182 is already newer than 182 2023-03-29T14:10:10.693 INFO:tasks.rados.rados.0.smithi017.stdout:1898: oid 18 version 182 is already newer than 182 2023-03-29T14:10:10.694 INFO:tasks.rados.rados.0.smithi017.stdout:1898: oid 18 version 182 is already newer than 182 2023-03-29T14:10:10.694 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 18 v 182 (ObjNum 905 snap 221 seq_num 905) dirty exists 2023-03-29T14:10:10.694 INFO:tasks.rados.rados.0.smithi017.stdout:1898: left oid 18 (ObjNum 905 snap 221 seq_num 905) 2023-03-29T14:10:10.694 INFO:tasks.rados.rados.0.smithi017.stdout:1902: finishing write tid 1 to smithi017107884-27 2023-03-29T14:10:10.694 INFO:tasks.rados.rados.0.smithi017.stdout:1900: finishing write tid 1 to smithi017107884-1 2023-03-29T14:10:10.694 INFO:tasks.rados.rados.0.smithi017.stdout:1901: expect (ObjNum 894 snap 217 seq_num 894) 2023-03-29T14:10:10.812 INFO:tasks.rados.rados.0.smithi017.stdout:1900: finishing write tid 2 to smithi017107884-1 2023-03-29T14:10:10.812 INFO:tasks.rados.rados.0.smithi017.stdout:1900: finishing write tid 3 to smithi017107884-1 2023-03-29T14:10:10.812 INFO:tasks.rados.rados.0.smithi017.stdout:1900: finishing write tid 4 to smithi017107884-1 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1900: finishing write tid 5 to smithi017107884-1 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1900: oid 1 updating version 0 to 273 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1900: oid 1 version 273 is already newer than 273 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1900: oid 1 version 273 is already newer than 273 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1900: oid 1 version 273 is already newer than 273 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 1 v 273 (ObjNum 906 snap 221 seq_num 906) dirty exists 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1900: left oid 1 (ObjNum 906 snap 221 seq_num 906) 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1903: finishing write tid 1 to smithi017107884-46 2023-03-29T14:10:10.813 INFO:tasks.rados.rados.0.smithi017.stdout:1903: finishing write tid 2 to smithi017107884-46 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: finishing write tid 3 to smithi017107884-46 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: finishing write tid 4 to smithi017107884-46 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: finishing write tid 5 to smithi017107884-46 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: oid 46 updating version 0 to 238 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: oid 46 version 238 is already newer than 234 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: oid 46 version 238 is already newer than 238 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: oid 46 version 238 is already newer than 238 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 46 v 238 (ObjNum 908 snap 221 seq_num 908) dirty exists 2023-03-29T14:10:10.814 INFO:tasks.rados.rados.0.smithi017.stdout:1903: left oid 46 (ObjNum 908 snap 221 seq_num 908) 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: finishing write tid 2 to smithi017107884-27 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: finishing write tid 3 to smithi017107884-27 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: finishing write tid 4 to smithi017107884-27 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: finishing write tid 5 to smithi017107884-27 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: oid 27 updating version 0 to 228 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: oid 27 updating version 228 to 231 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: oid 27 version 231 is already newer than 231 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:1902: oid 27 version 231 is already newer than 231 2023-03-29T14:10:10.815 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 27 v 231 (ObjNum 907 snap 221 seq_num 907) dirty exists 2023-03-29T14:10:10.816 INFO:tasks.rados.rados.0.smithi017.stdout:1902: left oid 27 (ObjNum 907 snap 221 seq_num 907) 2023-03-29T14:10:11.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:10:11.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:11.170+0000 7facd3edc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:11.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:11.174+0000 7facd3edc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:11.240 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.2 2023-03-29T14:10:11.242 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-03-29T14:10:11.243 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:10:11.408 INFO:tasks.rados.rados.0.smithi017.stdout:1898: done (6 left) 2023-03-29T14:10:11.409 INFO:tasks.rados.rados.0.smithi017.stdout:1899: done (5 left) 2023-03-29T14:10:11.409 INFO:tasks.rados.rados.0.smithi017.stdout:1900: done (4 left) 2023-03-29T14:10:11.409 INFO:tasks.rados.rados.0.smithi017.stdout:1901: done (3 left) 2023-03-29T14:10:11.409 INFO:tasks.rados.rados.0.smithi017.stdout:1902: done (2 left) 2023-03-29T14:10:11.409 INFO:tasks.rados.rados.0.smithi017.stdout:1903: done (1 left) 2023-03-29T14:10:11.409 INFO:tasks.rados.rados.0.smithi017.stdout:1904: done (0 left) 2023-03-29T14:10:11.409 INFO:tasks.rados.rados.0.smithi017.stdout:1905: delete oid 8 current snap is 222 2023-03-29T14:10:11.470 INFO:tasks.rados.rados.0.smithi017.stdout:1905: done (0 left) 2023-03-29T14:10:11.470 INFO:tasks.rados.rados.0.smithi017.stdout:1906: read oid 7 snap -1 2023-03-29T14:10:11.471 INFO:tasks.rados.rados.0.smithi017.stdout:1906: expect (ObjNum 807 snap 196 seq_num 807) 2023-03-29T14:10:11.471 INFO:tasks.rados.rados.0.smithi017.stdout:1907: read oid 50 snap 216 2023-03-29T14:10:11.471 INFO:tasks.rados.rados.0.smithi017.stdout:1907: expect deleted 2023-03-29T14:10:11.471 INFO:tasks.rados.rados.0.smithi017.stdout:1908: snap_remove snap 215 2023-03-29T14:10:12.070 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:10:12.070 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:10:12.070 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:10:12.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:12.198+0000 7f03a5be7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:12.204 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:12.204+0000 7f03a5be7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:12.404 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-03-29T14:10:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-03-29T14:10:12.412 INFO:tasks.rados.rados.0.smithi017.stdout:1908: done (2 left) 2023-03-29T14:10:12.412 INFO:tasks.rados.rados.0.smithi017.stdout:1909: delete oid 14 current snap is 222 2023-03-29T14:10:12.413 INFO:tasks.rados.rados.0.smithi017.stdout:1906: expect (ObjNum 807 snap 196 seq_num 807) 2023-03-29T14:10:12.419 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":701,"stamp":"2023-03-29T14:10:12.189720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":213769862,"num_objects":1072,"num_object_clones":976,"num_object_copies":3216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":4306,"num_read_kb":2559637,"num_write":5832,"num_write_kb":1620704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3091,"ondisk_log_size":3091,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":2016236,"kb_used_data":1928268,"kb_used_omap":0,"kb_used_meta":87936,"kb_avail":375471124,"statfs":{"total":386547056640,"available":384482430976,"internally_reserved":0,"allocated":1974546432,"data_stored":2459787290,"data_compressed":89556,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":90046464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8908304,"num_objects":10,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-2,"num_read":36,"num_read_kb":16288,"num_write":56,"num_write_kb":16866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000791"},"pg_stats":[{"pgid":"3.f","version":"458'138","reported_seq":470,"reported_epoch":464,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:10.414623+0000","last_change":"2023-03-29T14:05:43.098160+0000","last_active":"2023-03-29T14:10:10.414623+0000","last_peered":"2023-03-29T14:10:10.414623+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.457299+0000","last_became_peered":"2023-03-29T14:05:30.457299+0000","last_unstale":"2023-03-29T14:10:10.414623+0000","last_undegraded":"2023-03-29T14:05:43.097377+0000","last_fullsized":"2023-03-29T14:05:30.333881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11026184,"num_objects":32,"num_object_clones":32,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":71837,"num_write":190,"num_write_kb":56080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":32}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"409'60","reported_seq":483,"reported_epoch":463,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:09.458447+0000","last_change":"2023-03-29T14:05:30.557532+0000","last_active":"2023-03-29T14:10:09.458447+0000","last_peered":"2023-03-29T14:10:09.458447+0000","last_clean":"2023-03-29T14:05:29.273566+0000","last_became_active":"2023-03-29T14:05:30.557532+0000","last_became_peered":"2023-03-29T14:05:30.557532+0000","last_unstale":"2023-03-29T14:10:09.458447+0000","last_undegraded":"2023-03-29T14:05:30.373022+0000","last_fullsized":"2023-03-29T14:05:30.371891+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6924814,"num_objects":24,"num_object_clones":22,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":39146,"num_write":112,"num_write_kb":33664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"451'271","reported_seq":1004,"reported_epoch":464,"state":"active+clean","last_fresh":"2023-03-29T14:10:10.418244+0000","last_change":"2023-03-29T14:02:17.453555+0000","last_active":"2023-03-29T14:10:10.418244+0000","last_peered":"2023-03-29T14:10:10.418244+0000","last_clean":"2023-03-29T14:10:10.418244+0000","last_became_active":"2023-03-29T14:02:17.447025+0000","last_became_peered":"2023-03-29T14:02:17.447025+0000","last_unstale":"2023-03-29T14:10:10.418244+0000","last_undegraded":"2023-03-29T14:10:10.418244+0000","last_fullsized":"2023-03-29T14:10:10.418244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11765560,"num_objects":102,"num_object_clones":92,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":426,"num_read_kb":302859,"num_write":512,"num_write_kb":152358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"459'308","reported_seq":1064,"reported_epoch":464,"state":"active+clean","last_fresh":"2023-03-29T14:10:10.424973+0000","last_change":"2023-03-29T14:02:17.423462+0000","last_active":"2023-03-29T14:10:10.424973+0000","last_peered":"2023-03-29T14:10:10.424973+0000","last_clean":"2023-03-29T14:10:10.424973+0000","last_became_active":"2023-03-29T14:02:17.417453+0000","last_became_peered":"2023-03-29T14:02:17.417453+0000","last_unstale":"2023-03-29T14:10:10.424973+0000","last_undegraded":"2023-03-29T14:10:10.424973+0000","last_fullsized":"2023-03-29T14:10:10.424973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23733068,"num_objects":122,"num_object_clones":114,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":418,"num_read_kb":243749,"num_write":606,"num_write_kb":171374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"463'238","reported_seq":904,"reported_epoch":463,"state":"active+clean","last_fresh":"2023-03-29T14:10:10.741633+0000","last_change":"2023-03-29T14:02:17.495061+0000","last_active":"2023-03-29T14:10:10.741633+0000","last_peered":"2023-03-29T14:10:10.741633+0000","last_clean":"2023-03-29T14:10:10.741633+0000","last_became_active":"2023-03-29T14:02:17.485903+0000","last_became_peered":"2023-03-29T14:02:17.485903+0000","last_unstale":"2023-03-29T14:10:10.741633+0000","last_undegraded":"2023-03-29T14:10:10.741633+0000","last_fullsized":"2023-03-29T14:10:10.741633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10958890,"num_objects":86,"num_object_clones":78,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":4,"num_read":371,"num_read_kb":246694,"num_write":470,"num_write_kb":138714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":392,"reported_epoch":464,"state":"active+undersized","last_fresh":"2023-03-29T14:10:10.412683+0000","last_change":"2023-03-29T14:05:30.476341+0000","last_active":"2023-03-29T14:10:10.412683+0000","last_peered":"2023-03-29T14:10:10.412683+0000","last_clean":"2023-03-29T14:05:23.227099+0000","last_became_active":"2023-03-29T14:05:30.476341+0000","last_became_peered":"2023-03-29T14:05:30.476341+0000","last_unstale":"2023-03-29T14:10:10.412683+0000","last_undegraded":"2023-03-29T14:10:10.412683+0000","last_fullsized":"2023-03-29T14:05:30.349529+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"463'182","reported_seq":762,"reported_epoch":464,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:10.570708+0000","last_change":"2023-03-29T14:05:30.506334+0000","last_active":"2023-03-29T14:10:10.570708+0000","last_peered":"2023-03-29T14:10:10.570708+0000","last_clean":"2023-03-29T14:05:23.222820+0000","last_became_active":"2023-03-29T14:05:30.506334+0000","last_became_peered":"2023-03-29T14:05:30.506334+0000","last_unstale":"2023-03-29T14:10:10.570708+0000","last_undegraded":"2023-03-29T14:05:30.348433+0000","last_fullsized":"2023-03-29T14:05:30.347901+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12236288,"num_objects":64,"num_object_clones":58,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":188,"num_read_kb":80685,"num_write":328,"num_write_kb":93216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":64}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"449'108","reported_seq":633,"reported_epoch":464,"state":"active+clean","last_fresh":"2023-03-29T14:10:11.478859+0000","last_change":"2023-03-29T14:02:17.537102+0000","last_active":"2023-03-29T14:10:11.478859+0000","last_peered":"2023-03-29T14:10:11.478859+0000","last_clean":"2023-03-29T14:10:11.478859+0000","last_became_active":"2023-03-29T14:02:17.532805+0000","last_became_peered":"2023-03-29T14:02:17.532805+0000","last_unstale":"2023-03-29T14:10:11.478859+0000","last_undegraded":"2023-03-29T14:10:11.478859+0000","last_fullsized":"2023-03-29T14:10:11.478859+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12908502,"num_objects":38,"num_object_clones":34,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":103549,"num_write":202,"num_write_kb":57294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":359,"reported_epoch":463,"state":"active+undersized","last_fresh":"2023-03-29T14:10:09.443345+0000","last_change":"2023-03-29T14:05:30.528803+0000","last_active":"2023-03-29T14:10:09.443345+0000","last_peered":"2023-03-29T14:10:09.443345+0000","last_clean":"2023-03-29T14:05:29.255294+0000","last_became_active":"2023-03-29T14:05:30.528803+0000","last_became_peered":"2023-03-29T14:05:30.528803+0000","last_unstale":"2023-03-29T14:10:09.443345+0000","last_undegraded":"2023-03-29T14:10:09.443345+0000","last_fullsized":"2023-03-29T14:05:30.353101+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"456'113","reported_seq":398,"reported_epoch":463,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:09.603546+0000","last_change":"2023-03-29T14:06:36.372576+0000","last_active":"2023-03-29T14:10:09.603546+0000","last_peered":"2023-03-29T14:10:09.603546+0000","last_clean":"2023-03-29T14:02:16.192094+0000","last_became_active":"2023-03-29T14:05:30.480618+0000","last_became_peered":"2023-03-29T14:05:30.480618+0000","last_unstale":"2023-03-29T14:10:09.603546+0000","last_undegraded":"2023-03-29T14:06:36.370677+0000","last_fullsized":"2023-03-29T14:05:30.387172+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4672742,"num_objects":26,"num_object_clones":26,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":97,"num_read_kb":52640,"num_write":146,"num_write_kb":39178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":395,"reported_epoch":464,"state":"active+undersized","last_fresh":"2023-03-29T14:10:10.426455+0000","last_change":"2023-03-29T14:05:30.485021+0000","last_active":"2023-03-29T14:10:10.426455+0000","last_peered":"2023-03-29T14:10:10.426455+0000","last_clean":"2023-03-29T14:05:23.224969+0000","last_became_active":"2023-03-29T14:05:30.485021+0000","last_became_peered":"2023-03-29T14:05:30.485021+0000","last_unstale":"2023-03-29T14:10:10.426455+0000","last_undegraded":"2023-03-29T14:10:10.426455+0000","last_fullsized":"2023-03-29T14:05:30.351158+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"434'123","reported_seq":661,"reported_epoch":463,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:09.599259+0000","last_change":"2023-03-29T14:05:30.556006+0000","last_active":"2023-03-29T14:10:09.599259+0000","last_peered":"2023-03-29T14:10:09.599259+0000","last_clean":"2023-03-29T14:05:29.238493+0000","last_became_active":"2023-03-29T14:05:30.556006+0000","last_became_peered":"2023-03-29T14:05:30.556006+0000","last_unstale":"2023-03-29T14:10:09.599259+0000","last_undegraded":"2023-03-29T14:05:30.375514+0000","last_fullsized":"2023-03-29T14:05:30.373865+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9614396,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":199,"num_read_kb":133887,"num_write":246,"num_write_kb":72486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":220,"reported_epoch":463,"state":"active+undersized","last_fresh":"2023-03-29T14:10:09.455298+0000","last_change":"2023-03-29T14:05:30.543645+0000","last_active":"2023-03-29T14:10:09.455298+0000","last_peered":"2023-03-29T14:10:09.455298+0000","last_clean":"2023-03-29T14:02:09.178132+0000","last_became_active":"2023-03-29T14:05:30.543645+0000","last_became_peered":"2023-03-29T14:05:30.543645+0000","last_unstale":"2023-03-29T14:10:09.455298+0000","last_undegraded":"2023-03-29T14:10:09.455298+0000","last_fullsized":"2023-03-29T14:05:30.364979+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"426'161","reported_seq":727,"reported_epoch":463,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:09.468342+0000","last_change":"2023-03-29T14:05:30.630460+0000","last_active":"2023-03-29T14:10:09.468342+0000","last_peered":"2023-03-29T14:10:09.468342+0000","last_clean":"2023-03-29T14:05:29.259690+0000","last_became_active":"2023-03-29T14:05:30.630460+0000","last_became_peered":"2023-03-29T14:05:30.630460+0000","last_unstale":"2023-03-29T14:10:09.468342+0000","last_undegraded":"2023-03-29T14:05:30.405254+0000","last_fullsized":"2023-03-29T14:05:30.403094+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530328,"num_objects":60,"num_object_clones":54,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":257,"num_read_kb":155096,"num_write":306,"num_write_kb":90700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":395,"reported_epoch":464,"state":"active+undersized","last_fresh":"2023-03-29T14:10:10.425539+0000","last_change":"2023-03-29T14:05:30.522439+0000","last_active":"2023-03-29T14:10:10.425539+0000","last_peered":"2023-03-29T14:10:10.425539+0000","last_clean":"2023-03-29T14:05:23.215732+0000","last_became_active":"2023-03-29T14:05:30.522439+0000","last_became_peered":"2023-03-29T14:05:30.522439+0000","last_unstale":"2023-03-29T14:10:10.425539+0000","last_undegraded":"2023-03-29T14:10:10.425539+0000","last_fullsized":"2023-03-29T14:05:30.358384+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'97","reported_seq":584,"reported_epoch":463,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:09.456583+0000","last_change":"2023-03-29T14:05:30.598930+0000","last_active":"2023-03-29T14:10:09.456583+0000","last_peered":"2023-03-29T14:10:09.456583+0000","last_clean":"2023-03-29T14:05:29.256399+0000","last_became_active":"2023-03-29T14:05:30.598930+0000","last_became_peered":"2023-03-29T14:05:30.598930+0000","last_unstale":"2023-03-29T14:10:09.456583+0000","last_undegraded":"2023-03-29T14:05:30.390754+0000","last_fullsized":"2023-03-29T14:05:30.389192+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:08.175335+0000","last_clean_scrub_stamp":"2023-03-29T14:02:08.175335+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"463'231","reported_seq":946,"reported_epoch":464,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:10.809870+0000","last_change":"2023-03-29T14:05:30.470054+0000","last_active":"2023-03-29T14:10:10.809870+0000","last_peered":"2023-03-29T14:10:10.809870+0000","last_clean":"2023-03-29T14:05:23.227628+0000","last_became_active":"2023-03-29T14:05:30.470054+0000","last_became_peered":"2023-03-29T14:05:30.470054+0000","last_unstale":"2023-03-29T14:10:10.809870+0000","last_undegraded":"2023-03-29T14:05:30.346348+0000","last_fullsized":"2023-03-29T14:05:30.345784+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17446854,"num_objects":86,"num_object_clones":78,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":399,"num_read_kb":264975,"num_write":448,"num_write_kb":129388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":86}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":395,"reported_epoch":464,"state":"active+undersized","last_fresh":"2023-03-29T14:10:10.422091+0000","last_change":"2023-03-29T14:05:30.466454+0000","last_active":"2023-03-29T14:10:10.422091+0000","last_peered":"2023-03-29T14:10:10.422091+0000","last_clean":"2023-03-29T14:05:23.217263+0000","last_became_active":"2023-03-29T14:05:30.466454+0000","last_became_peered":"2023-03-29T14:05:30.466454+0000","last_unstale":"2023-03-29T14:10:10.422091+0000","last_undegraded":"2023-03-29T14:10:10.422091+0000","last_fullsized":"2023-03-29T14:05:30.336123+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"457'398","reported_seq":1259,"reported_epoch":463,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:09.612660+0000","last_change":"2023-03-29T14:05:30.461951+0000","last_active":"2023-03-29T14:10:09.612660+0000","last_peered":"2023-03-29T14:10:09.612660+0000","last_clean":"2023-03-29T14:05:29.220683+0000","last_became_active":"2023-03-29T14:05:30.461951+0000","last_became_peered":"2023-03-29T14:05:30.461951+0000","last_unstale":"2023-03-29T14:10:09.612660+0000","last_undegraded":"2023-03-29T14:05:30.355069+0000","last_fullsized":"2023-03-29T14:05:30.353881+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22926332,"num_objects":146,"num_object_clones":132,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":6,"num_read":585,"num_read_kb":355985,"num_write":740,"num_write_kb":216430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":359,"reported_epoch":463,"state":"active+undersized","last_fresh":"2023-03-29T14:10:09.435632+0000","last_change":"2023-03-29T14:05:30.578627+0000","last_active":"2023-03-29T14:10:09.435632+0000","last_peered":"2023-03-29T14:10:09.435632+0000","last_clean":"2023-03-29T14:05:29.252396+0000","last_became_active":"2023-03-29T14:05:30.578627+0000","last_became_peered":"2023-03-29T14:05:30.578627+0000","last_unstale":"2023-03-29T14:10:09.435632+0000","last_undegraded":"2023-03-29T14:10:09.435632+0000","last_fullsized":"2023-03-29T14:05:30.397133+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"442'128","reported_seq":637,"reported_epoch":463,"state":"active+clean","last_fresh":"2023-03-29T14:10:09.433426+0000","last_change":"2023-03-29T14:02:17.547506+0000","last_active":"2023-03-29T14:10:09.433426+0000","last_peered":"2023-03-29T14:10:09.433426+0000","last_clean":"2023-03-29T14:10:09.433426+0000","last_became_active":"2023-03-29T14:02:17.537100+0000","last_became_peered":"2023-03-29T14:02:17.537100+0000","last_unstale":"2023-03-29T14:10:09.433426+0000","last_undegraded":"2023-03-29T14:10:09.433426+0000","last_fullsized":"2023-03-29T14:10:09.433426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10421132,"num_objects":46,"num_object_clones":42,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":102553,"num_write":260,"num_write_kb":76088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"459'262","reported_seq":910,"reported_epoch":463,"state":"active+clean","last_fresh":"2023-03-29T14:10:10.646240+0000","last_change":"2023-03-29T14:02:17.569226+0000","last_active":"2023-03-29T14:10:10.646240+0000","last_peered":"2023-03-29T14:10:10.646240+0000","last_clean":"2023-03-29T14:10:10.646240+0000","last_became_active":"2023-03-29T14:02:17.556220+0000","last_became_peered":"2023-03-29T14:02:17.556220+0000","last_unstale":"2023-03-29T14:10:10.646240+0000","last_undegraded":"2023-03-29T14:10:10.646240+0000","last_fullsized":"2023-03-29T14:10:10.646240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27599008,"num_objects":90,"num_object_clones":80,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":321,"num_read_kb":181128,"num_write":476,"num_write_kb":136836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":359,"reported_epoch":463,"state":"active+undersized","last_fresh":"2023-03-29T14:10:09.453255+0000","last_change":"2023-03-29T14:05:30.611315+0000","last_active":"2023-03-29T14:10:09.453255+0000","last_peered":"2023-03-29T14:10:09.453255+0000","last_clean":"2023-03-29T14:05:29.242166+0000","last_became_active":"2023-03-29T14:05:30.611315+0000","last_became_peered":"2023-03-29T14:05:30.611315+0000","last_unstale":"2023-03-29T14:10:09.453255+0000","last_undegraded":"2023-03-29T14:10:09.453255+0000","last_fullsized":"2023-03-29T14:05:30.408699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":395,"reported_epoch":464,"state":"active+undersized","last_fresh":"2023-03-29T14:10:10.416385+0000","last_change":"2023-03-29T14:05:30.511247+0000","last_active":"2023-03-29T14:10:10.416385+0000","last_peered":"2023-03-29T14:10:10.416385+0000","last_clean":"2023-03-29T14:05:23.212348+0000","last_became_active":"2023-03-29T14:05:30.511247+0000","last_became_peered":"2023-03-29T14:05:30.511247+0000","last_unstale":"2023-03-29T14:10:10.416385+0000","last_undegraded":"2023-03-29T14:10:10.416385+0000","last_fullsized":"2023-03-29T14:05:30.353942+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:09.178132+0000","last_clean_scrub_stamp":"2023-03-29T14:02:09.178132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"463'273","reported_seq":970,"reported_epoch":463,"state":"active+undersized+degraded","last_fresh":"2023-03-29T14:10:10.568206+0000","last_change":"2023-03-29T14:05:30.541330+0000","last_active":"2023-03-29T14:10:10.568206+0000","last_peered":"2023-03-29T14:10:10.568206+0000","last_clean":"2023-03-29T14:05:29.211098+0000","last_became_active":"2023-03-29T14:05:30.541330+0000","last_became_peered":"2023-03-29T14:05:30.541330+0000","last_unstale":"2023-03-29T14:10:10.568206+0000","last_undegraded":"2023-03-29T14:05:30.365485+0000","last_fullsized":"2023-03-29T14:05:30.364291+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T14:02:16.192094+0000","last_clean_scrub_stamp":"2023-03-29T14:02:16.192094+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":80,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18825028,"num_objects":100,"num_object_clones":92,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":365,"num_read_kb":224734,"num_write":516,"num_write_kb":153110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":100}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":212589126,"num_objects":1068,"num_object_clones":976,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":26,"num_read":4164,"num_read_kb":2559517,"num_write":5558,"num_write_kb":1616916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2994,"ondisk_log_size":2994,"up":38,"acting":38,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":1180736,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":120,"num_write":274,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":598184,"kb_used_data":573536,"kb_used_omap":0,"kb_used_meta":24640,"kb_avail":93773656,"statfs":{"total":96636764160,"available":96024223744,"internally_reserved":0,"allocated":587300864,"data_stored":733103891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25231360},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155944,"kb_used_data":143840,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94215896,"statfs":{"total":96636764160,"available":96477077504,"internally_reserved":0,"allocated":147292160,"data_stored":173232225,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":9,"seq":38654706148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":660504,"kb_used_data":634384,"kb_used_omap":0,"kb_used_meta":26112,"kb_avail":93711336,"statfs":{"total":96636764160,"available":95960408064,"internally_reserved":0,"allocated":649609216,"data_stored":813921937,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":26738688},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":9,"seq":38654706148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":601604,"kb_used_data":576508,"kb_used_omap":0,"kb_used_meta":25088,"kb_avail":93770236,"statfs":{"total":96636764160,"available":96020721664,"internally_reserved":0,"allocated":590344192,"data_stored":739529237,"data_compressed":29852,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":25690112},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-03-29T14:10:12.422 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.9 3.3 1.0 3.6 2023-03-29T14:10:12.559 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:12.550+0000 7fd826943700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:12.562 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:12.562+0000 7fd826943700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:12.595 INFO:tasks.rados.rados.0.smithi017.stdout:1906: done (2 left) 2023-03-29T14:10:12.595 INFO:tasks.rados.rados.0.smithi017.stdout:1907: done (1 left) 2023-03-29T14:10:12.595 INFO:tasks.rados.rados.0.smithi017.stdout:1909: done (0 left) 2023-03-29T14:10:12.595 INFO:tasks.rados.rados.0.smithi017.stdout:1910: write oid 31 current snap is 222 2023-03-29T14:10:12.595 INFO:tasks.rados.rados.0.smithi017.stdout:1910: seq_num 911 ranges {511177=797846,1976390=591639,3034464=276713} 2023-03-29T14:10:12.602 INFO:tasks.rados.rados.0.smithi017.stdout:1910: writing smithi017107884-31 from 511177 to 1309023 tid 1 2023-03-29T14:10:12.609 INFO:tasks.rados.rados.0.smithi017.stdout:1910: writing smithi017107884-31 from 1976390 to 2568029 tid 2 2023-03-29T14:10:12.614 INFO:tasks.rados.rados.0.smithi017.stdout:1910: writing smithi017107884-31 from 3034464 to 3311177 tid 3 2023-03-29T14:10:12.614 INFO:tasks.rados.rados.0.smithi017.stdout:1911: snap_remove snap 219 2023-03-29T14:10:12.775 INFO:teuthology.orchestra.run.smithi017.stderr:instructing pg(s) [3.3,3.6] on osd.0 to force-recovery; instructing pg(s) [1.0] on osd.1 to force-recovery; instructing pg(s) [3.9] on osd.3 to force-recovery; 2023-03-29T14:10:12.775 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-03-29T14:10:12.789 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~303s 2023-03-29T14:10:12.789 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-03-29T14:10:13.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:10:13.271 INFO:tasks.rados.rados.0.smithi017.stdout:1911: done (1 left) 2023-03-29T14:10:13.272 INFO:tasks.rados.rados.0.smithi017.stdout:1912: write (excl) oid 48 current snap is 222 2023-03-29T14:10:13.272 INFO:tasks.rados.rados.0.smithi017.stdout:1910: finishing write tid 1 to smithi017107884-31 2023-03-29T14:10:13.272 INFO:tasks.rados.rados.0.smithi017.stdout:1912: seq_num 912 ranges {527984=473095,1634872=451650,2661887=634311} 2023-03-29T14:10:13.277 INFO:tasks.rados.rados.0.smithi017.stdout:1912: writing smithi017107884-48 from 527984 to 1001079 tid 1 2023-03-29T14:10:13.283 INFO:tasks.rados.rados.0.smithi017.stdout:1912: writing smithi017107884-48 from 1634872 to 2086522 tid 2 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1912: writing smithi017107884-48 from 2661887 to 3296198 tid 3 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1913: write (excl) oid 5 current snap is 222 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: finishing write tid 2 to smithi017107884-31 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: finishing write tid 3 to smithi017107884-31 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: finishing write tid 4 to smithi017107884-31 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: finishing write tid 5 to smithi017107884-31 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: oid 31 updating version 0 to 188 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: oid 31 version 188 is already newer than 188 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: oid 31 version 188 is already newer than 188 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: oid 31 version 188 is already newer than 188 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:update_object_version oid 31 v 188 (ObjNum 911 snap 222 seq_num 911) dirty exists 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1910: left oid 31 (ObjNum 911 snap 222 seq_num 911) 2023-03-29T14:10:13.290 INFO:tasks.rados.rados.0.smithi017.stdout:1913: seq_num 913 ranges {544791=664697,1693354=595308,2689310=445583} 2023-03-29T14:10:13.293 INFO:tasks.ceph.osd.0:Sent signal 15 2023-03-29T14:10:13.294 INFO:tasks.ceph.osd.1:Sent signal 15 2023-03-29T14:10:13.294 INFO:tasks.ceph.osd.3:Sent signal 15 2023-03-29T14:10:13.295 INFO:tasks.ceph.mon.a:Sent signal 15 2023-03-29T14:10:13.295 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-03-29T14:10:13.296 INFO:tasks.rados.rados.0.smithi017.stdout:1913: writing smithi017107884-5 from 544791 to 1209488 tid 1 2023-03-29T14:10:13.297 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:10:13.297+0000 7fcae057c700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 105436) UID: 0 2023-03-29T14:10:13.297 INFO:tasks.ceph.mgr.x.smithi089.stderr:2023-03-29T14:10:13.297+0000 7fcae057c700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-03-29T14:10:13.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2023-03-29T14:10:13.296+0000 7f892471e700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 105950) UID: 0 2023-03-29T14:10:13.297 INFO:tasks.ceph.mon.a.smithi017.stderr:2023-03-29T14:10:13.296+0000 7f892471e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-03-29T14:10:13.302 INFO:tasks.rados.rados.0.smithi017.stdout:1913: writing smithi017107884-5 from 1693354 to 2288662 tid 2 2023-03-29T14:10:13.307 INFO:tasks.rados.rados.0.smithi017.stdout:1913: writing smithi017107884-5 from 2689310 to 3134893 tid 3 2023-03-29T14:10:13.307 INFO:tasks.rados.rados.0.smithi017.stdout:1910: done (2 left) 2023-03-29T14:10:13.308 INFO:tasks.rados.rados.0.smithi017.stdout:1914: read oid 8 snap -1 2023-03-29T14:10:13.308 INFO:tasks.rados.rados.0.smithi017.stdout:1914: expect deleted 2023-03-29T14:10:13.308 INFO:tasks.rados.rados.0.smithi017.stdout:1915: read oid 31 snap -1 2023-03-29T14:10:13.308 INFO:tasks.rados.rados.0.smithi017.stdout:1915: expect (ObjNum 911 snap 222 seq_num 911) 2023-03-29T14:10:13.308 INFO:tasks.rados.rados.0.smithi017.stdout:1916: snap_remove snap 221 2023-03-29T14:10:13.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:13.374+0000 7f5e1c98a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:10:17.787 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2023-03-29T14:10:17.787 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2023-03-29T14:10:17.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T14:10:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:10:17.916+0000 7fcdd9ec8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:12:13.278 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:12:13.278 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:12:15.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:12:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:12:15.407+0000 7fb760b8f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:12:17.821 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:12:17.823 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 190, in wrapper return func(self) File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 1401, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 780, in force_cancel_recovery self.cancel_force_recovery() File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 763, in cancel_force_recovery j = self.ceph_manager.get_pgids_to_cancel_force(backfill) File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 2381, in get_pgids_to_cancel_force j = self.get_pg_stats(); File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 2334, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 1588, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi017 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-03-29T14:12:17.824 ERROR:tasks.thrashosds.thrasher:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 1269, in do_thrash self._do_thrash() File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 190, in wrapper return func(self) File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 1401, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 780, in force_cancel_recovery self.cancel_force_recovery() File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 763, in cancel_force_recovery j = self.ceph_manager.get_pgids_to_cancel_force(backfill) File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 2381, in get_pgids_to_cancel_force j = self.get_pg_stats(); File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 2334, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 1588, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_6c07bf89555b2fa3413019059913f7a1fb12fde5/qa/tasks/ceph_manager.py", line 1579, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e3729161eecf8e6d09a4069acbb883cdcd5e9f50/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi017 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-03-29T14:14:15.312 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:14:15.313 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:14:17.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:14:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:14:17.442+0000 7fc6f6434700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:16:17.346 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:16:17.347 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:16:19.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:16:19.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:16:19.477+0000 7fb7a63d5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:18:19.382 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:18:19.383 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:18:21.383 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:18:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:18:21.512+0000 7f2c330fe700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:20:21.417 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:20:21.418 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:20:23.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:20:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:20:23.548+0000 7fe36e32c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:22:23.453 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:22:23.454 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:22:25.455 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:22:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:22:25.582+0000 7f848cca8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:24:25.488 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:24:25.489 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:24:27.490 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:24:27.619 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:24:27.617+0000 7f93e1cfa700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:26:27.522 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:26:27.523 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:26:29.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:26:29.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:26:29.652+0000 7f85c1fda700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:28:29.558 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:28:29.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:28:31.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:28:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:28:31.689+0000 7fdbb0ed3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:30:31.594 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:30:31.596 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:30:33.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:30:33.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:30:33.725+0000 7f4a548e2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:32:33.630 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:32:33.631 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:32:35.631 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:32:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:32:35.760+0000 7fca90dee700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:34:35.665 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:34:35.666 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:34:37.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:34:37.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:34:37.795+0000 7f6bd12bf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:36:37.700 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:36:37.700 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:36:39.702 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:36:39.831 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:36:39.829+0000 7f6199195700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:38:39.734 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:38:39.735 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:38:41.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:38:41.866 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:38:41.865+0000 7fdee13c4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:40:41.771 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:40:41.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:40:43.773 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:40:43.903 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:40:43.900+0000 7fa7a60c7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:42:43.806 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:42:43.807 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:42:45.809 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:42:45.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:42:45.935+0000 7f09a1f46700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:44:45.842 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:44:45.842 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:44:47.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:44:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:44:47.970+0000 7ff7b1d2d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:46:47.877 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:46:47.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:46:49.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:46:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:46:50.007+0000 7f1fb6ada700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:48:49.913 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:48:49.914 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:48:51.915 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:48:52.049 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:48:52.048+0000 7fa81bcc8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:50:51.953 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:50:51.954 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:50:53.955 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:50:54.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:50:54.086+0000 7f6e40f67700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:52:53.990 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:52:53.991 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:52:55.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:52:56.123 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:52:56.122+0000 7f2baacd3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:54:56.026 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:54:56.026 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:54:58.028 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:54:58.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:54:58.157+0000 7f1c729da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:56:58.060 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:56:58.061 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:57:00.063 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T14:57:00.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:57:00.192+0000 7f66affde700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T14:59:00.095 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T14:59:00.097 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T14:59:02.097 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T14:59:02.227 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T14:59:02.225+0000 7f07d0d1e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:01:02.131 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:01:02.131 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:01:04.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:01:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:01:04.262+0000 7fa784437700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:03:04.167 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:03:04.168 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:03:06.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:03:06.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:03:06.297+0000 7ff3199b0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:05:06.202 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:05:06.202 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:05:08.203 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:05:08.332 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:05:08.332+0000 7faa68402700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:07:08.236 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:07:08.237 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:07:10.238 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:07:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:07:10.367+0000 7f7e858e9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:09:10.271 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:09:10.272 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:09:12.273 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:09:12.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:09:12.403+0000 7f96ed866700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:11:12.307 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:11:12.307 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:11:14.309 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:11:14.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:11:14.438+0000 7f27deb02700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:13:14.343 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:13:14.344 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:13:16.345 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:13:16.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:13:16.476+0000 7f66b3438700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:15:16.379 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:15:16.380 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:15:18.381 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:15:18.511 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:15:18.509+0000 7f709b17d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:17:18.414 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:17:18.415 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:17:20.416 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:17:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:17:20.543+0000 7f8349dd0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:19:20.449 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:19:20.450 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:19:22.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:19:22.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:19:22.579+0000 7f13acd32700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:21:22.485 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:21:22.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:21:24.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:21:24.618 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:21:24.618+0000 7f4d4e1b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:23:24.522 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:23:24.524 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:23:26.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:23:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:23:26.654+0000 7fc808619700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:25:26.558 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:25:26.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:25:28.560 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:25:28.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:25:28.689+0000 7f73d7b66700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:27:28.594 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:27:28.595 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:27:30.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:27:30.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:27:30.724+0000 7fb8fa5d9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:29:30.629 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:29:30.630 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:29:32.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:29:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:29:32.759+0000 7f11945d8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:31:32.664 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:31:32.665 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:31:34.666 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:31:34.798 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:31:34.797+0000 7fe5c498d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:33:34.702 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:33:34.702 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:33:36.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:33:36.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:33:36.833+0000 7f52984b6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:35:36.738 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:35:36.739 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:35:38.741 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:35:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:35:38.871+0000 7fccdb30a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:37:38.776 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:37:38.776 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:37:40.777 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:37:40.908 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:37:40.906+0000 7fc408ade700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:39:40.812 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:39:40.813 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:39:42.814 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:39:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:39:42.945+0000 7fa6b7bc5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:41:42.850 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:41:42.851 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:41:44.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:41:44.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:41:44.980+0000 7fb895ee3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:43:44.885 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:43:44.886 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:43:46.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:43:47.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:43:47.016+0000 7f2e69833700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:45:46.920 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:45:46.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:45:48.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:45:49.055 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:45:49.054+0000 7f3699638700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:47:48.959 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:47:48.960 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:47:50.961 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:47:51.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:47:51.089+0000 7f72265b2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:49:50.994 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:49:50.995 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:49:52.996 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:49:53.126 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:49:53.124+0000 7f373cde3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:51:53.030 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:51:53.031 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:51:55.033 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:51:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:51:55.164+0000 7efc33154700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:53:55.071 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:53:55.071 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:53:57.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:53:57.202 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:53:57.200+0000 7f30bc05b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:55:57.106 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:55:57.107 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:55:59.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T15:55:59.239 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:55:59.238+0000 7fe070878700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T15:57:59.143 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T15:57:59.144 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T15:58:01.145 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T15:58:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T15:58:01.276+0000 7fb5ca689700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:00:01.180 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:00:01.181 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:00:03.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:00:03.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:00:03.312+0000 7f4299267700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:02:03.217 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:02:03.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:02:05.219 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:02:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:02:05.348+0000 7f669b862700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:04:05.254 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:04:05.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:04:07.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:04:07.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:04:07.386+0000 7f07ad104700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:06:07.293 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:06:07.293 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:06:09.294 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:06:09.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:06:09.423+0000 7fe04f14f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:08:09.330 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:08:09.331 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:08:11.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:08:11.462 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:08:11.460+0000 7f9bca9ed700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:10:11.366 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:10:11.367 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:10:13.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:10:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:10:13.499+0000 7f0380d77700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:12:13.404 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:12:13.404 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:12:15.406 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:12:15.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:12:15.536+0000 7f50b0132700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:14:15.442 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:14:15.442 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:14:17.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:14:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:14:17.574+0000 7fbefee5b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:16:17.479 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:16:17.480 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:16:19.480 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:16:19.610 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:16:19.608+0000 7f24fadd4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:18:19.513 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:18:19.514 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:18:21.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:18:21.646 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:18:21.644+0000 7fcc451f1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:20:21.551 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:20:21.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:20:23.553 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:20:23.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:20:23.683+0000 7f66e0cc5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:22:23.589 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:22:23.590 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:22:25.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:22:25.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:22:25.721+0000 7feaf9f79700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:24:25.627 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:24:25.628 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:24:27.629 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:24:27.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:24:27.759+0000 7f762188c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:26:27.665 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:26:27.666 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:26:29.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:26:29.796 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:26:29.795+0000 7fb882214700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:28:29.701 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:28:29.702 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:28:31.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:28:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:28:31.831+0000 7f7022558700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:30:31.736 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:30:31.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:30:33.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:30:33.866 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:30:33.865+0000 7fe4ff573700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:32:33.770 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:32:33.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:32:35.773 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:32:35.903 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:32:35.902+0000 7fe38e6af700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:34:35.806 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:34:35.807 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:34:37.808 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:34:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:34:37.937+0000 7f518491d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:36:37.842 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:36:37.843 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:36:39.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:36:39.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:36:39.974+0000 7f278286e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:38:39.878 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:38:39.879 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:38:41.880 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:38:42.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:38:42.007+0000 7f3e77b49700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:40:41.913 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:40:41.914 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:40:43.915 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:40:44.046 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:40:44.044+0000 7fe795b99700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:42:43.950 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:42:43.951 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:42:45.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:42:46.081 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:42:46.080+0000 7f14707c5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:44:45.986 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:44:45.987 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:44:47.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:44:48.118 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:44:48.118+0000 7f4984dd5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:46:48.024 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:46:48.025 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:46:50.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:46:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:46:50.157+0000 7fa6abfb0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:48:50.062 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:48:50.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:48:52.063 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:48:52.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:48:52.193+0000 7f33355e9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:50:52.096 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:50:52.097 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:50:54.098 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:50:54.227 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:50:54.226+0000 7f404d27c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:52:54.131 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:52:54.132 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:52:56.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:52:56.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:52:56.263+0000 7f06f8577700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:54:56.166 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:54:56.167 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:54:58.168 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:54:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:54:58.297+0000 7fc9b0638700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:56:58.201 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:56:58.203 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:57:00.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T16:57:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:57:00.336+0000 7f2addf65700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T16:59:00.243 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T16:59:00.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T16:59:02.245 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T16:59:02.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T16:59:02.373+0000 7f1e9d8a8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:01:02.278 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:01:02.279 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:01:04.280 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:01:04.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:01:04.414+0000 7f1c6c99f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:03:04.318 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:03:04.319 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:03:06.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:03:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:03:06.450+0000 7f00b0d05700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:05:06.353 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:05:06.354 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:05:08.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:05:08.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:05:08.485+0000 7f7c8fe7e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:07:08.389 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:07:08.390 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:07:10.391 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:07:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:07:10.520+0000 7f64a9475700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:09:10.425 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:09:10.425 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:09:12.426 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:09:12.557 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:09:12.557+0000 7f9d79440700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:11:12.461 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:11:12.462 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:11:14.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:11:14.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:11:14.596+0000 7f94f6e83700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:13:14.502 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:13:14.503 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:13:16.503 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:13:16.633 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:13:16.632+0000 7fc7ff3d3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:15:16.536 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:15:16.537 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:15:18.538 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:15:18.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:15:18.666+0000 7f0d1a6e4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:17:18.572 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:17:18.573 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:17:20.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:17:20.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:17:20.705+0000 7f0dcae5c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:19:20.610 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:19:20.611 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:19:22.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:19:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:19:22.744+0000 7ff880540700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:21:22.648 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:21:22.649 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:21:24.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:21:24.780 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:21:24.780+0000 7f0519372700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:23:24.683 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:23:24.685 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:23:26.686 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:23:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:23:26.815+0000 7f4361771700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:25:26.719 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:25:26.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:25:28.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:25:28.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:25:28.852+0000 7f36f806e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:27:28.757 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:27:28.758 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:27:30.759 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:27:30.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:27:30.888+0000 7ff1da8d0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:29:30.792 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:29:30.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:29:32.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:29:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:29:32.922+0000 7f158f5c8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:31:32.828 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:31:32.829 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:31:34.832 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:31:34.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:31:34.962+0000 7fc88a883700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:33:34.865 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:33:34.866 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:33:36.867 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:33:36.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:33:36.995+0000 7fb18a547700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:35:36.900 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:35:36.901 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:35:38.902 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:35:39.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:35:39.033+0000 7f239c716700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:37:38.938 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:37:38.958 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:37:40.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:37:41.091 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:37:41.091+0000 7f2b5b90b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:39:40.994 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:39:40.995 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:39:42.997 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:39:43.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:39:43.128+0000 7fbde2c26700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:41:43.032 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:41:43.033 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:41:45.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:41:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:41:45.166+0000 7f4ea1e8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:43:45.070 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:43:45.071 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:43:47.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:43:47.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:43:47.205+0000 7f34e8e0f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:45:47.108 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:45:47.109 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:45:49.109 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:45:49.238 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:45:49.238+0000 7f8eba68a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:47:49.142 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:47:49.143 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:47:51.144 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:47:51.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:47:51.274+0000 7ff84aebd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:49:51.179 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:49:51.179 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:49:53.181 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:49:53.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:49:53.311+0000 7fa3207be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:51:53.217 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:51:53.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:51:55.219 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:51:55.349 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:51:55.348+0000 7f0f173bf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:53:55.252 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:53:55.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:53:57.254 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:53:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:53:57.386+0000 7f7150658700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:55:57.291 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:55:57.291 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:55:59.293 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T17:55:59.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:55:59.424+0000 7f5a6445a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T17:57:59.328 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T17:57:59.328 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T17:58:01.330 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T17:58:01.463 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T17:58:01.461+0000 7feccbaac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:00:01.365 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:00:01.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:00:03.367 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:00:03.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:00:03.498+0000 7ff1c2130700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:02:03.402 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:02:03.403 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:02:05.404 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:02:05.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:02:05.538+0000 7f099bb6d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:04:05.441 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:04:05.442 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:04:07.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:04:07.574 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:04:07.575+0000 7f170a427700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:06:07.478 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:06:07.480 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:06:09.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:06:09.612 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:06:09.611+0000 7f292bb22700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:08:09.517 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:08:09.517 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:08:11.518 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:08:11.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:08:11.646+0000 7f0dddfad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:10:11.551 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:10:11.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:10:13.552 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:10:13.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:10:13.684+0000 7fea9ad2d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:12:13.588 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:12:13.589 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:12:15.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:12:15.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:12:15.724+0000 7f80f3434700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:14:15.626 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:14:15.626 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:14:17.627 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:14:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:14:17.758+0000 7f4a0a4ff700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:16:17.663 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:16:17.664 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:16:19.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:16:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:16:19.795+0000 7f0271125700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:18:19.700 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:18:19.701 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:18:21.701 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:18:21.832 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:18:21.832+0000 7f12b29f2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:20:21.737 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:20:21.738 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:20:23.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:20:23.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:20:23.871+0000 7fda2ed9b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:22:23.775 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:22:23.776 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:22:25.777 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:22:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:22:25.905+0000 7f54106f8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:24:25.810 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:24:25.812 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:24:27.812 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:24:27.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:24:27.940+0000 7f588e2be700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:26:27.846 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:26:27.847 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:26:29.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:26:29.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:26:29.977+0000 7ff42baf1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:28:29.883 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:28:29.884 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:28:31.885 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:28:32.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:28:32.015+0000 7fd1edb67700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:30:31.920 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:30:31.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:30:33.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:30:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:30:34.052+0000 7f22625ab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:32:33.957 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:32:33.958 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:32:35.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:32:36.091 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:32:36.090+0000 7f1feb35f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:34:35.994 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:34:35.996 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:34:37.996 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:34:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:34:38.126+0000 7f2a849ec700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:36:38.031 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:36:38.032 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:36:40.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:36:40.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:36:40.161+0000 7f55e06da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:38:40.066 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:38:40.067 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:38:42.068 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:38:42.200 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:38:42.197+0000 7f64b7d77700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:40:42.103 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:40:42.103 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:40:44.105 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:40:44.237 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:40:44.234+0000 7f4f3cf9b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:42:44.141 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:42:44.141 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:42:46.142 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:42:46.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:42:46.273+0000 7ff500c91700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:44:46.178 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:44:46.179 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:44:48.180 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:44:48.313 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:44:48.311+0000 7f1ed6c8c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:46:48.217 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:46:48.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:46:50.219 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:46:50.347 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:46:50.346+0000 7ff3a85b8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:48:50.251 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:48:50.252 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:48:52.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:48:52.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:48:52.383+0000 7ff614811700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:50:52.289 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:50:52.290 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:50:54.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:50:54.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:50:54.419+0000 7fad7fae3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:52:54.325 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:52:54.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:52:56.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:52:56.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:52:56.458+0000 7ff9a8957700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:54:56.365 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:54:56.366 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:54:58.367 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:54:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:54:58.494+0000 7f9066773700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:56:58.400 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:56:58.400 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:57:00.401 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T18:57:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:57:00.529+0000 7f523a332700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T18:59:00.436 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T18:59:00.436 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T18:59:02.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T18:59:02.569 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T18:59:02.566+0000 7f77fe4cc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:01:02.473 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:01:02.474 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:01:04.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:01:04.607 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:01:04.605+0000 7f23705ee700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:03:04.512 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:03:04.512 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:03:06.513 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:03:06.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:03:06.642+0000 7fe56f509700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:05:06.547 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:05:06.548 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:05:08.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:05:08.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:05:08.680+0000 7f906d8cd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:07:08.585 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:07:08.658 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:07:10.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:07:10.789 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:07:10.787+0000 7f6ef640a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:09:10.692 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:09:10.693 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:09:12.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:09:12.825 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:09:12.824+0000 7fcaaa73d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:11:12.727 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:11:12.789 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:11:14.790 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:11:14.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:11:14.921+0000 7fb9d947d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:13:14.828 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:13:14.829 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:13:16.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:13:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:13:16.961+0000 7efde651e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:15:16.866 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:15:17.003 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:15:19.004 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:15:19.135 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:15:19.134+0000 7fc5b2c8f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:17:19.039 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:17:19.040 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:17:21.041 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:17:21.172 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:17:21.170+0000 7f2beb155700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:19:21.075 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:19:21.076 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:19:23.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:19:23.208 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:19:23.206+0000 7fbcce676700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:21:23.112 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:21:23.112 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:21:25.113 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:21:25.243 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:21:25.240+0000 7fc039359700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:23:25.146 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:23:25.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:23:27.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:23:27.280 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:23:27.279+0000 7f9be0659700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:25:27.183 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:25:27.184 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:25:29.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:25:29.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:25:29.314+0000 7f7deb5e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:27:29.218 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:27:29.219 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:27:31.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:27:31.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:27:31.351+0000 7f3e2d42a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:29:31.255 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:29:31.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:29:33.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:29:33.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:29:33.389+0000 7f7784485700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:31:33.292 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:31:33.292 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:31:35.294 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:31:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:31:35.426+0000 7efe8142a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:33:35.330 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:33:35.331 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:33:37.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:33:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:33:37.463+0000 7fa4e6b98700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:35:37.367 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:35:37.368 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:35:39.369 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:35:39.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:35:39.500+0000 7f408255f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:37:39.405 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:37:39.406 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:37:41.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:37:41.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:37:41.538+0000 7f8b543a9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:39:41.442 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:39:41.442 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:39:43.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:39:43.575 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:39:43.575+0000 7f0ed8201700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:41:43.478 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:41:43.479 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:41:45.480 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:41:45.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:41:45.612+0000 7f2d786ac700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:43:45.515 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:43:45.516 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:43:47.517 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:43:47.649 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:43:47.650+0000 7f95e2063700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:45:47.552 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:45:47.553 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:45:49.554 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:45:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:45:49.688+0000 7f11339ad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:47:49.590 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:47:49.590 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:47:51.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:47:51.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:47:51.723+0000 7fd99fbf7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:49:51.627 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:49:51.627 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:49:53.628 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:49:53.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:49:53.758+0000 7f4aae3ea700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:51:53.662 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:51:53.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:51:55.679 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:51:55.812 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:51:55.812+0000 7f5a780f0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:53:55.716 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:53:55.717 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:53:57.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:53:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:53:57.847+0000 7f25ff0a5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:55:57.751 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:55:57.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:55:59.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T19:55:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:55:59.885+0000 7faffb141700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T19:57:59.790 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T19:57:59.791 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T19:58:01.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T19:58:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T19:58:01.924+0000 7f2df6eab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:00:01.828 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:00:01.829 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:00:03.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:00:03.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:00:03.959+0000 7f26db56f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:02:03.864 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:02:03.865 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:02:05.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:02:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:02:06.127+0000 7f5ef71de700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:04:06.031 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:04:06.031 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:04:08.032 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:04:08.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:04:08.163+0000 7fde18576700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:06:08.068 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:06:08.069 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:06:10.070 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:06:10.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:06:10.197+0000 7f9cae0db700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:08:10.104 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:08:10.104 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:08:12.105 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:08:12.237 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:08:12.235+0000 7fb23c237700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:10:12.140 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:10:12.141 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:10:14.143 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:10:14.286 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:10:14.285+0000 7f4f4338c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:12:14.190 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:12:14.190 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:12:16.191 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:12:16.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:12:16.321+0000 7f2801ad2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:14:16.228 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:14:16.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:14:18.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:14:18.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:14:18.357+0000 7ff392459700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:16:18.263 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:16:18.264 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:16:20.264 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:16:20.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:16:20.395+0000 7f6f238cb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:18:20.301 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:18:20.302 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:18:22.302 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:18:22.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:18:22.428+0000 7fcc9f3b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:20:22.336 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:20:22.336 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:20:24.337 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:20:24.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:20:24.466+0000 7fc4f623c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:22:24.373 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:22:24.374 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:22:26.375 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:22:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:22:26.504+0000 7f0d34269700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:24:26.411 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:24:26.412 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:24:28.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:24:28.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:24:28.542+0000 7f1950e68700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:26:28.448 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:26:28.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:26:30.450 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:26:30.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:26:30.579+0000 7fd16e6e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:28:30.485 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:28:30.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:28:32.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:28:32.618 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:28:32.616+0000 7fc10e3fc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:30:32.522 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:30:32.523 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:30:34.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:30:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:30:34.652+0000 7f874bb78700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:32:34.557 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:32:34.557 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:32:36.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:32:36.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:32:36.686+0000 7f0fbc4ee700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:34:36.592 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:34:36.593 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:34:38.594 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:34:38.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:34:38.721+0000 7f93c5d30700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:36:38.626 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:36:38.627 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:36:40.627 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:36:40.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:36:40.754+0000 7fb8b107b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:38:40.661 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:38:40.662 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:38:42.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:38:42.792 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:38:42.789+0000 7fba83314700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:40:42.695 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:40:42.697 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:40:44.697 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:40:44.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:40:44.824+0000 7fcc0873c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:42:44.730 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:42:44.731 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:42:46.732 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:42:46.863 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:42:46.862+0000 7f9b44b1c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:44:46.768 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:44:46.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:44:48.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:44:48.901 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:44:48.900+0000 7fc16aeb0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:46:48.805 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:46:48.806 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:46:50.807 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:46:50.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:46:50.938+0000 7f5de9c20700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:48:50.845 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:48:50.846 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:48:52.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:48:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:48:52.975+0000 7f8e058ab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:50:52.881 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:50:52.881 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:50:54.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:50:55.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:50:55.009+0000 7f1906187700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:52:54.916 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:52:54.918 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:52:56.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:52:57.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:52:57.045+0000 7fbbcb9a0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:54:56.952 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:54:56.967 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:54:58.969 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:54:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:54:59.096+0000 7f6624522700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:56:59.003 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:56:59.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:57:01.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T20:57:01.136 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:57:01.134+0000 7f130cc96700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T20:59:01.040 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T20:59:01.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T20:59:03.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T20:59:03.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T20:59:03.172+0000 7fe32f116700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:01:03.079 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:01:03.080 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:01:05.081 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:01:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:01:05.212+0000 7f8492886700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:03:05.119 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:03:05.119 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:03:07.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:03:07.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:03:07.248+0000 7fcc89055700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:05:07.153 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:05:07.154 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:05:09.155 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:05:09.287 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:05:09.286+0000 7fc33f895700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:07:09.191 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:07:09.192 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:07:11.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:07:11.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:07:11.323+0000 7f4e9ddcc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:09:11.229 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:09:11.251 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:09:13.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:09:13.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:09:13.382+0000 7f456ce71700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:11:13.288 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:11:13.289 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:11:15.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:11:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:11:15.422+0000 7f1066c89700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:13:15.327 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:13:15.328 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:13:17.329 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:13:17.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:13:17.459+0000 7fe120df6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:15:17.364 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:15:17.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:15:19.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:15:19.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:15:19.494+0000 7f6c3b567700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:17:19.399 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:17:19.442 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:17:21.444 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:17:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:17:21.575+0000 7f3658d8d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:19:21.480 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:19:21.481 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:19:23.482 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:19:23.615 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:19:23.614+0000 7fdec8140700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:21:23.519 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:21:23.519 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:21:25.520 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:21:25.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:21:25.651+0000 7f0984c86700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:23:25.555 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:23:25.571 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:23:27.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:23:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:23:27.699+0000 7f357fb70700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:25:27.604 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:25:27.605 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:25:29.607 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:25:29.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:25:29.734+0000 7f5f14874700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:27:29.641 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:27:29.642 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:27:31.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:27:31.774 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:27:31.773+0000 7f342f887700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:29:31.678 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:29:31.679 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:29:33.680 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:29:33.809 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:29:33.808+0000 7f4ab6833700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:31:33.713 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:31:33.714 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:31:35.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:31:35.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:31:35.846+0000 7f75bb485700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:33:35.752 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:33:35.753 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:33:37.754 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:33:37.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:33:37.883+0000 7f7e1e01c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:35:37.789 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:35:37.790 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:35:39.791 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:35:39.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:35:39.922+0000 7f48826b2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:37:39.828 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:37:39.829 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:37:41.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:37:41.961 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:37:41.958+0000 7fabd30b3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:39:41.865 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:39:41.866 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:39:43.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:39:43.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:39:43.995+0000 7f6f58093700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:41:43.902 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:41:43.903 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:41:45.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:41:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:41:46.035+0000 7f7e5ee4d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:43:45.942 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:43:45.943 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:43:47.944 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:43:48.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:43:48.071+0000 7fadc0251700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:45:47.977 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:45:47.978 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:45:49.979 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:45:50.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:45:50.108+0000 7fcbca324700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:47:50.016 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:47:50.017 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:47:52.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:47:52.147 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:47:52.143+0000 7f20ac11b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:49:52.051 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:49:52.051 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:49:54.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:49:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:49:54.181+0000 7fc23e6fb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:51:54.089 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:51:54.090 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:51:56.091 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:51:56.219 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:51:56.214+0000 7fec30ec5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:53:56.124 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:53:56.125 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:53:58.126 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:53:58.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:53:58.250+0000 7f2e3b0b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:55:58.159 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:55:58.160 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:56:00.161 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T21:56:00.294 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:56:00.290+0000 7f2ad730f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T21:58:00.198 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T21:58:00.199 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T21:58:02.201 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T21:58:02.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T21:58:02.327+0000 7f019ede0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:00:02.235 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:00:02.236 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:00:04.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:00:04.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:00:04.362+0000 7f2f1e533700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:02:04.271 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:02:04.272 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:02:06.274 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:02:06.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:02:06.400+0000 7f6746258700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:04:06.308 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:04:06.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:04:08.317 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:04:08.448 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:04:08.443+0000 7efc6a5a4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:06:08.353 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:06:08.354 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:06:10.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:06:10.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:06:10.478+0000 7fba113ab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:08:10.389 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:08:10.390 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:08:12.391 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:08:12.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:08:12.513+0000 7f8afdaee700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:10:12.423 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:10:12.424 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:10:14.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:10:14.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:10:14.549+0000 7f1255e4c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:12:14.458 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:12:14.459 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:12:16.460 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:12:16.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:12:16.585+0000 7fafe0ffd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:14:16.493 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:14:16.513 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:14:18.514 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:14:18.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:14:18.639+0000 7f1948a81700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:16:18.549 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:16:18.549 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:16:20.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:16:20.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:16:20.674+0000 7f29a18b6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:18:20.583 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:18:20.584 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:18:22.585 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:18:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:18:22.708+0000 7f6f9ffe0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:20:22.618 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:20:22.619 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:20:24.621 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:20:24.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:20:24.748+0000 7efd6a9a9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:22:24.658 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:22:24.659 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:22:26.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:22:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:22:26.784+0000 7f2eccd57700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:24:26.694 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:24:26.694 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:24:28.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:24:28.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:24:28.819+0000 7f50d62e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:26:28.730 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:26:28.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:26:30.749 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:26:30.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:26:30.871+0000 7f01d0c6f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:28:30.782 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:28:30.783 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:28:32.784 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:28:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:28:32.907+0000 7f129982a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:30:32.817 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:30:32.818 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:30:34.819 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:30:34.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:30:34.943+0000 7f09c7941700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:32:34.853 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:32:34.854 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:32:36.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:32:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:32:36.978+0000 7fcf8ce8c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:34:36.888 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:34:36.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:34:38.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:34:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:34:39.011+0000 7fb157f5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:36:38.923 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:36:38.924 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:36:40.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:36:41.057 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:36:41.050+0000 7fbefe165700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:38:40.961 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:38:40.962 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:38:42.963 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:38:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:38:43.084+0000 7fd1ce3a2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:40:42.996 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:40:43.003 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:40:45.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:40:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:40:45.127+0000 7fd9089ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:42:45.038 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:42:45.039 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:42:47.040 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:42:47.172 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:42:47.165+0000 7f2c326d9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:44:47.076 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:44:47.078 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:44:49.078 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:44:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:44:49.203+0000 7f6c549e8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:46:49.114 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:46:49.115 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:46:51.116 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:46:51.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:46:51.241+0000 7f9a78445700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:48:51.151 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:48:51.153 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:48:53.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:48:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:48:53.281+0000 7f6d80345700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:50:53.191 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:50:53.192 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:50:55.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:50:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:50:55.318+0000 7f6b285b9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:52:55.225 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:52:55.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:52:57.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:52:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:52:57.354+0000 7f972a5c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:54:57.261 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:54:57.262 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:54:59.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:54:59.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:54:59.391+0000 7fca24fd0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:56:59.299 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:56:59.299 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:57:01.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T22:57:01.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:57:01.426+0000 7fd5bde36700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T22:59:01.333 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T22:59:01.334 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T22:59:03.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T22:59:03.468 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T22:59:03.466+0000 7f464fe83700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:01:03.372 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:01:03.373 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:01:05.374 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:01:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:01:05.502+0000 7f2c4c2b1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:03:05.408 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:03:05.408 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:03:07.409 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:03:07.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:03:07.539+0000 7f829142c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:05:07.444 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:05:07.445 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:05:09.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:05:09.577 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:05:09.575+0000 7f5b7ac57700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:07:09.481 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:07:09.482 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:07:11.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:07:11.614 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:07:11.611+0000 7f8126211700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:09:11.518 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:09:11.519 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:09:13.519 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:09:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:09:13.648+0000 7fea90d5e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:11:13.554 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:11:13.576 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:11:15.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:11:15.710 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:11:15.708+0000 7f42cb91c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:13:15.613 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:13:15.614 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:13:17.615 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:13:17.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:13:17.743+0000 7f3d9d3c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:15:17.648 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:15:17.649 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:15:19.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:15:19.782 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:15:19.780+0000 7f9de1698700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:17:19.686 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:17:19.686 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:17:21.688 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:17:21.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:17:21.817+0000 7f5dbfd09700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:19:21.722 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:19:21.723 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:19:23.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:19:23.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:19:23.852+0000 7fa6767d0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:21:23.758 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:21:23.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:21:25.760 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:21:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:21:25.889+0000 7fb1283c6700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:23:25.796 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:23:25.797 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:23:27.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:23:27.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:23:27.925+0000 7f224f053700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:25:27.834 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:25:27.856 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:25:29.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:25:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:25:29.986+0000 7fbb5f588700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:27:29.893 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:27:29.893 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:27:31.894 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:27:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:27:32.021+0000 7f941dd9f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:29:31.929 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:29:31.930 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:29:33.932 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:29:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:29:34.057+0000 7f6824e88700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:31:33.964 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:31:33.965 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:31:35.966 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:31:36.095 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:31:36.093+0000 7fee7602b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:33:36.000 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:33:36.000 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:33:38.002 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:33:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:33:38.131+0000 7f43c38c3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:35:38.037 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:35:38.038 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:35:40.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:35:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:35:40.168+0000 7f43bf441700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:37:40.075 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:37:40.075 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:37:42.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:37:42.206 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:37:42.202+0000 7fab1ed49700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:39:42.110 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:39:42.111 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:39:44.112 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:39:44.244 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:39:44.240+0000 7fc35c260700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:41:44.148 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:41:44.148 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:41:46.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:41:46.280 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:41:46.277+0000 7eff953da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:43:46.183 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:43:46.184 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:43:48.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:43:48.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:43:48.313+0000 7f6967ebf700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:45:48.220 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:45:48.221 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:45:50.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:45:50.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:45:50.348+0000 7f22f8a02700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:47:50.255 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:47:50.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:47:52.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:47:52.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:47:52.384+0000 7fc57a18a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:49:52.290 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:49:52.291 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:49:54.293 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:49:54.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:49:54.418+0000 7f516e5cd700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:51:54.326 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:51:54.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:51:56.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:51:56.457 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:51:56.453+0000 7f66c5ba7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:53:56.360 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:53:56.362 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:53:58.363 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:53:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:53:58.489+0000 7f36f50d1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:55:58.395 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:55:58.396 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:56:00.397 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-29T23:56:00.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:56:00.523+0000 7f044e591700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-29T23:58:00.429 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-29T23:58:00.430 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-29T23:58:02.431 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-29T23:58:02.560 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-29T23:58:02.557+0000 7fafe6819700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:00:02.464 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:00:02.465 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:00:04.466 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:00:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:00:04.594+0000 7fc6c62a4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:02:04.500 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:02:04.502 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:02:06.503 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:02:06.636 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:02:06.633+0000 7f9ea3d6a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:04:06.539 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:04:06.539 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:04:08.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:04:08.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:04:08.667+0000 7faea09da700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:06:08.574 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:06:08.575 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:06:10.576 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:06:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:06:10.704+0000 7f7575151700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:08:10.610 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:08:10.611 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:08:12.613 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:08:12.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:08:12.740+0000 7f71680a8700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:10:12.651 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:10:12.652 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:10:14.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:10:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:10:14.780+0000 7f479a8e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:12:14.686 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:12:14.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:12:16.688 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:12:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:12:16.815+0000 7f85a065d700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:14:16.722 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:14:16.723 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:14:18.723 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:14:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:14:18.849+0000 7f75fda9b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:16:18.755 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:16:18.757 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:16:20.757 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:16:20.886 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:16:20.883+0000 7f102f4ad700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:18:20.790 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:18:20.791 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:18:22.792 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:18:22.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:18:22.919+0000 7f00acf5c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:20:22.825 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:20:22.826 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:20:24.827 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:20:24.957 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:20:24.955+0000 7f41a60fb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:22:24.860 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:22:24.861 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:22:26.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:22:26.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:22:26.991+0000 7fa0f2afb700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:24:26.896 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:24:26.897 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:24:28.899 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:24:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:24:29.028+0000 7fe69ac92700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:26:28.933 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:26:28.934 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:26:30.936 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:26:31.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:26:31.064+0000 7f4c3bda9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:28:30.969 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:28:30.970 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:28:32.971 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:28:33.100 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:28:33.098+0000 7f5b7927a700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:30:33.004 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:30:33.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:30:35.007 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:30:35.136 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:30:35.132+0000 7f1221371700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:32:35.040 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:32:35.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:32:37.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:32:37.172 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:32:37.169+0000 7f0887411700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:34:37.075 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:34:37.076 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:34:39.078 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:34:39.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:34:39.203+0000 7f52d23b7700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:36:39.111 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:36:39.112 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:36:41.113 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:36:41.243 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:36:41.240+0000 7f2e2dd1f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:38:41.146 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:38:41.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:38:43.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:38:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:38:43.274+0000 7f5b8dd31700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:40:43.181 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:40:43.183 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:40:45.184 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:40:45.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:40:45.311+0000 7f468afc1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:42:45.218 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:42:45.219 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:42:47.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:42:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:42:47.349+0000 7f5c57bef700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:44:47.255 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:44:47.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:44:49.258 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:44:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:44:49.384+0000 7f1ca389c700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:46:49.291 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:46:49.292 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:46:51.294 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:46:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:46:51.421+0000 7f4c401b5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:48:51.327 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:48:51.328 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:48:53.329 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:48:53.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:48:53.456+0000 7fcec5a44700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:50:53.364 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:50:53.377 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:50:55.379 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:50:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:50:55.505+0000 7fac102e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:52:55.413 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:52:55.414 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:52:57.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:52:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:52:57.541+0000 7eff419e3700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:54:57.449 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:54:57.450 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:54:59.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:54:59.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:54:59.578+0000 7f7fd40e1700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:56:59.485 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:56:59.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:57:01.488 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T00:57:01.618 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:57:01.615+0000 7f20f63e5700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T00:59:01.521 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T00:59:01.522 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T00:59:03.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T00:59:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T00:59:03.650+0000 7f14a1ff2700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:01:03.558 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:01:03.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:01:05.560 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:01:05.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:01:05.689+0000 7fb7a5a57700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:03:05.594 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:03:05.595 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:03:07.597 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:03:07.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:03:07.724+0000 7ff75755b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:05:07.630 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:05:07.631 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:05:09.631 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:05:09.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:05:09.758+0000 7fc7d71ed700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:07:09.664 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:07:09.665 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:07:11.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:07:11.795 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:07:11.791+0000 7fd686344700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:09:11.698 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:09:11.699 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:09:13.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:09:13.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:09:13.828+0000 7f3b941ab700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:11:13.733 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:11:13.734 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:11:15.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:11:15.865 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:11:15.863+0000 7f580b05e700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:13:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:13:15.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:13:17.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:13:17.899 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:13:17.897+0000 7f0e43850700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:15:17.803 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:15:17.804 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:15:19.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:15:19.934 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:15:19.933+0000 7fc4d7de4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:17:19.839 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:17:19.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:17:21.841 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:17:21.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:17:21.969+0000 7f85afd5f700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:19:21.874 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:19:21.875 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:19:23.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:19:24.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:19:24.003+0000 7f5995e08700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:21:23.911 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:21:23.912 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:21:25.913 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:21:26.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:21:26.041+0000 7f2d9a2b0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:23:25.947 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:23:25.948 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:23:27.949 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:23:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:23:28.076+0000 7f6999c13700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:25:27.983 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:25:27.984 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:25:29.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:25:30.115 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:25:30.114+0000 7ff6f4902700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:27:30.019 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:27:30.020 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:27:32.022 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:27:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:27:32.150+0000 7f8911e3b700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:29:32.055 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:29:32.056 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:29:34.057 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:29:34.187 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:29:34.186+0000 7f1e201d9700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:31:34.090 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:31:34.092 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:31:36.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:31:36.223 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:31:36.221+0000 7f8c4a5dc700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:33:36.125 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:33:36.126 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:33:38.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:33:38.256 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:33:38.254+0000 7f5be4756700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:35:38.161 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:35:38.162 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:35:40.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:35:40.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:35:40.296+0000 7f67925c4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:37:40.202 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:37:40.203 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:37:42.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:37:42.336 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:37:42.335+0000 7f352d737700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:39:42.240 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:39:42.240 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:39:44.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:39:44.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:39:44.371+0000 7f03923a4700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:41:44.277 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:41:44.278 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:41:46.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:41:46.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:41:46.409+0000 7fe0ebb01700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:43:46.314 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:43:46.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:43:48.316 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-03-30T01:43:48.448 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:43:48.446+0000 7f0aa93ce700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:45:48.351 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-03-30T01:45:48.352 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-03-30T01:45:50.353 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-03-30T01:45:50.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-03-30T01:45:50.483+0000 7f663f207700 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-03-30T01:47:36.417 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-03-30T01:47:36.418 DEBUG:teuthology.task.console_log:Killing console logger for smithi017 2023-03-30T01:47:36.418 DEBUG:teuthology.task.console_log:Killing console logger for smithi089 2023-03-30T01:47:36.418 DEBUG:teuthology.exit:Finished running handlers